Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error in pydantic when running python gradio_chat.py --baseonly #97

Open
sirredbeard opened this issue Sep 5, 2024 · 1 comment
Open

Error in pydantic when running python gradio_chat.py --baseonly #97

sirredbeard opened this issue Sep 5, 2024 · 1 comment

Comments

@sirredbeard
Copy link
Contributor

Following local finetuning README

Ran python gradio_chat.py --baseonly

Got:

(phi-3-env) hayden@XPS15:/mnt/d/phi-3-env/inference$ python gradio_chat.py --baseonly
Number of GPUs available: 1
Running on device: cuda
CPU threads: 10
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
`flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.
Current `flash-attention` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [05:08<00:00, 154.26s/it]
Number of GPUs available: 1
Model ../model-cache/microsoft/Phi-3-mini-4k-instruct loaded successfully on cuda
INFO:     Started server process [17975]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:7860 (Press CTRL+C to quit)
INFO:     127.0.0.1:59874 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/index-D6iiusuW.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/index-Ds_LdHYW.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/svelte/svelte.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Index-DvJ399W-.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Index-B4pUrfBk.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/_commonjsHelpers-BosuxZz1.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/index-Ds_LdHYW.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /info HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /theme.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Button-CTZL5Nos.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Blocks-xuLVTZz4.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Blocks-BrGSw8d-.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Button-BM3Gfoxv.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Index-COkUHsKJ.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Index-DPQUg1KC.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Index-CptIZeFZ.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Textbox-D8IAzrZj.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Index-B0JJ6p9c.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Example-Cj3ii62O.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Index-INgKs-Lw.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Index-CJeiS5ET.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Index-DPgNZtxV.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Textbox-OSHpBx5r.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/BlockTitle-BG3S3JH7.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Info-Cs8uP4Sq.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Check-Ck0iADAu.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Copy-ZPOKSMtK.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Index-BEdjhdex.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Example-C7XUkkid.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59888 - "GET /assets/Index-DDCF2BFd.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /assets/Index-D-Msc1SI.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:59874 - "GET /favicon.ico HTTP/1.1" 200 OK
INFO:     127.0.0.1:59898 - "POST /queue/join HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/type_adapter.py", line 277, in _init_core_attrs
    self._core_schema = _getattr_no_parents(self._type, '__pydantic_core_schema__')
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/type_adapter.py", line 119, in _getattr_no_parents
    raise AttributeError(attribute)
AttributeError: __pydantic_core_schema__

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 406, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
    return await self.app(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
    raise exc
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
    await app(scope, receive, sender)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/routing.py", line 460, in handle
    await self.app(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/gradio/route_utils.py", line 713, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/gradio/route_utils.py", line 729, in simple_response
    await self.app(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
    raise exc
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
    await app(scope, receive, sender)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
    raise exc
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
    await app(scope, receive, sender)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/starlette/routing.py", line 73, in app
    response = await f(request)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/fastapi/routing.py", line 291, in app
    solved_result = await solve_dependencies(
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/fastapi/dependencies/utils.py", line 639, in solve_dependencies
    ) = await request_body_to_args(  # body_params checked above
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/fastapi/dependencies/utils.py", line 810, in request_body_to_args
    fields_to_extract = get_model_fields(first_field.type_)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/fastapi/_compat.py", line 283, in get_model_fields
    return [
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/fastapi/_compat.py", line 284, in <listcomp>
    ModelField(field_info=field_info, name=name)
  File "<string>", line 6, in __init__
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/fastapi/_compat.py", line 109, in __post_init__
    self._type_adapter: TypeAdapter[Any] = TypeAdapter(
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/type_adapter.py", line 264, in __init__
    self._init_core_attrs(rebuild_mocks=False)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/type_adapter.py", line 142, in wrapped
    return func(self, *args, **kwargs)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/type_adapter.py", line 284, in _init_core_attrs
    self._core_schema = _get_schema(self._type, config_wrapper, parent_depth=self._parent_depth)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/type_adapter.py", line 102, in _get_schema
    schema = gen.generate_schema(type_)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 512, in generate_schema
    schema = self._generate_schema_inner(obj)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 768, in _generate_schema_inner
    return self._annotated_schema(obj)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1822, in _annotated_schema
    schema = self._apply_annotations(source_type, annotations)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1890, in _apply_annotations
    schema = get_inner_schema(source_type)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_schema_generation_shared.py", line 83, in __call__
    schema = self._handler(source_type)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1972, in new_handler
    schema = metadata_get_schema(source, get_inner_schema)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1968, in <lambda>
    lambda source, handler: handler(source)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_schema_generation_shared.py", line 83, in __call__
    schema = self._handler(source_type)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1871, in inner_handler
    schema = self._generate_schema_inner(obj)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 789, in _generate_schema_inner
    return self.match_type(obj)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 871, in match_type
    return self._match_generic_type(obj, origin)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 895, in _match_generic_type
    return self._union_schema(obj)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1207, in _union_schema
    choices.append(self.generate_schema(arg))
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 512, in generate_schema
    schema = self._generate_schema_inner(obj)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 789, in _generate_schema_inner
    return self.match_type(obj)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 875, in match_type
    return self._unknown_type_schema(obj)
  File "/opt/miniconda/envs/phi-3-env/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 415, in _unknown_type_schema
    raise PydanticSchemaGenerationError(
pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for <class 'starlette.requests.Request'>. Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.

If you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.

For further information visit https://errors.pydantic.dev/2.8/u/schema-for-unknown-type
@sirredbeard
Copy link
Contributor Author

Running on Ubuntu WSL 22.04

Seems to be an issue with starlette, see also #95

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@sirredbeard