GLiNER_Inference_Server/README.rst

GLiNER_Inference_Server
=========

GLiNER_Inference_Server - an inference server using FastAPI.

Quick start
-----------

1. Build the Docker image
~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

   docker build -t gliner-server .

2. Run the container
~~~~~~~~~~~~~~~~~~~~

.. code:: bash

   docker run -p 8000:8000 gliner-server

Use PORT enviroment variable to override port:

.. code:: bash

   docker run -p 9000:9000 -e PORT=9000 gliner-server

3. Health check
~~~~~~~~~~~~~~~

.. code:: bash

   curl http://localhost:8000/health

enviroment variables
--------------------

- ``PORT`` - server's port (default: 8000)
- ``MODEL_NAME`` - model name HuggingFace (default: knowledgator/gliner-multitask-large-v0.5)
- ``MAX_TEXT_LENGTH`` - maximum text length (default: 1000)

API Endpoints Examples
----------------------

1. ``/general`` - Named Entity Recognition
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

**Single request:**

.. code:: bash

   curl -X POST http://localhost:8000/general \
     -H "Content-Type: application/json" \
     -d '{
       "text": "Apple Inc. was founded by Steve Jobs in California.",
       "entities": ["organization", "person", "location"],
       "threshold": 0.5
     }'

**Batch request:**

.. code:: bash

   curl -X POST http://localhost:8000/general \
     -H "Content-Type: application/json" \
     -d '{
       "texts": [
         "Apple Inc. was founded by Steve Jobs in California.",
         "Microsoft is located in Redmond, Washington."
       ],
       "entities": ["organization", "person", "location"],
       "threshold": 0.5
     }'

**Batch request with `prompt`:**

.. code:: json

   curl -X POST http://localhost:8000/general \
     -H "Content-Type: application/json" \
     -d '{
       "texts": [
         "Apple Inc. was founded by Steve Jobs in California.",
         "Microsoft is located in Redmond, Washington."
       ],
       "entities": ["organization", "person", "location"],
       "prompt": "Identify organizations, people, and places from the text:\n",
       "threshold": 0.5
     }'

**Response:**

.. code:: json

   {
     "outputs": [
       {
         "entity": "organization",
         "span": "Apple Inc.",
         "start": 0,
         "end": 10,
         "score": 0.95
       },
       {
         "entity": "person",
         "span": "Steve Jobs",
         "start": 27,
         "end": 37,
         "score": 0.92
       }
     ]
   }

2. ``/relation-extraction`` - Extraction відносин
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

**Single request:**

.. code:: bash

   curl -X POST http://localhost:8000/relation-extraction \
     -H "Content-Type: application/json" \
     -d '{
       "text": "Steve Jobs founded Apple Inc. in California.",
       "relations": ["founded_by", "located_in"],
       "entities": ["person", "organization", "location"],
       "threshold": 0.5
     }'

**Batch request:**

.. code:: bash

   curl -X POST http://localhost:8000/relation-extraction \
     -H "Content-Type: application/json" \
     -d '{
       "texts": [
         "Steve Jobs founded Apple Inc. in California.",
         "Bill Gates leads Microsoft."
       ],
       "relations": ["founded_by", "leads"],
       "threshold": 0.5
     }'

**Batch request with prompt:**

.. code:: bash

   curl -X POST http://localhost:8000/relation-extraction \
     -H "Content-Type: application/json" \
     -d '{
       "texts": [
         "Steve Jobs founded Apple Inc. in California.",
         "Bill Gates leads Microsoft."
       ],
       "relations": ["founded_by", "leads"],
       "prompt": "Extract the relationships between the entities:\n",
       "threshold": 0.5
     }'

**Response:**

.. code:: json

   {
     "outputs": [
       {
         "source": {
           "entity": "organization",
           "span": "Apple Inc.",
           "start": 20,
           "end": 30,
           "score": 0.95
         },
         "relation": "founded_by",
         "target": {
           "entity": "person",
           "span": "Steve Jobs",
           "start": 0,
           "end": 10,
           "score": 0.92
         },
         "score": 0.88
       }
     ]
   }

3. ``/summarization`` - Summarization
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

**Single request:**

.. code:: bash

   curl -X POST http://localhost:8000/summarization \
     -H "Content-Type: application/json" \
     -d '{
       "text": "Artificial intelligence is transforming industries worldwide. Machine learning enables computers to learn from data. Deep learning uses neural networks for complex tasks.",
       "threshold": 0.5
     }'

**Batch request:**

.. code:: bash

   curl -X POST http://localhost:8000/summarization \
     -H "Content-Type: application/json" \
     -d '{
       "texts": [
         "First document about AI and ML...",
         "Second document about data science..."
       ],
       "threshold": 0.5
     }'

**Single request with `prompt`:**

.. code:: bash

   curl -X POST http://localhost:8000/summarization \
     -H "Content-Type: application/json" \
     -d '{
       "text": "Artificial intelligence is transforming industries worldwide. Machine learning enables computers to learn from data. Deep learning uses neural networks for complex tasks.",
       "prompt": "Generate a concise summary of the key concepts:\n",
       "threshold": 0.5
     }'

**Response:**

.. code:: json

   {
     "outputs": [
       {
         "text": "Artificial intelligence is transforming industries worldwide.",
         "start": 0,
         "end": 61,
         "score": 0.8586996793746948
       },
       {
         "text": "Machine learning enables computers to learn from data.",
         "start": 62,
         "end": 116,
         "score": 0.9330101013183594
       },
       {
         "text": "Deep learning uses neural networks for complex tasks.",
         "start": 117,
         "end": 170,
         "score": 0.8803190588951111
       }
     ]
   }

Error handling
--------------

All errors are returned in the standard format:

.. code:: json

   {
     "error": {
       "code": "INVALID_INPUT",
       "message": "Provide either 'text' or 'texts', not both",
       "details": {}
     }
   }

Error codes
~~~~~~~~~~~

- ``INVALID_INPUT`` (400) - invalid input data
- ``INTERNAL_ERROR`` (500) - internal server error

Validation
----------

- **text** or **texts** - one of them is mandatory (but not both)
- **entities/relations** - cannot be empty lists
- **threshold** - value from 0.0 to 1.0
- **texts** - cannot be an empty list
- The maximum length of the text is controlled through ``MAX_TEXT_LENGTH``

Author
======

Kostiantyn Klochko 2025