You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
44 lines
1.6 KiB
44 lines
1.6 KiB
1 month ago
|
|
||
|
|
**Цель:** собрать инструмент для обработки научных статей.
|
||
|
|
Работа в этом направлении: https://arxiv.org/abs/2210.02830
|
||
|
|
|
||
|
|
**Задачи**:
|
||
|
|
|
||
|
|
|
||
|
|
**Что будем использовать**
|
||
|
|
|
||
|
|
|
||
|
|
# Доп материалы
|
||
|
|
## Text embeddings
|
||
|
|
|
||
|
|
- https://qdrant.github.io/fastembed/
|
||
|
|
- https://github.com/qdrant/qdrant
|
||
|
|
|
||
|
|
## Библиотеки для парсинга
|
||
|
|
|
||
|
|
- https://github.com/Filimoa/open-parse/tree/main
|
||
|
|
- https://github.com/jsvine/pdfplumber
|
||
|
|
- https://github.com/topics/pdf-parser
|
||
|
|
- https://github.com/py-pdf/pypdf
|
||
|
|
- https://github.com/smalot/pdfparser
|
||
|
|
- https://github.com/jstockwin/py-pdf-parser
|
||
|
|
- https://github.com/RDFLib/rdflib
|
||
|
|
- https://pypi.org/project/camelot-py/
|
||
|
|
- https://pypi.org/project/tabula-py/
|
||
|
|
- apach-tika
|
||
|
|
|
||
|
|
**AI парсилки**
|
||
|
|
|
||
|
|
Здесь пример zero-shot pdf extraction на основе gpt-mini: https://github.com/getomni-ai/zerox?tab=readme-ov-file внутри есть ссылки на другие платные альтернативы:
|
||
|
|
- https://aws.amazon.com/textract/pricing/#:~:text=Amazon%20Textract%20API%20pricing
|
||
|
|
- https://cloud.google.com/document-ai/pricing
|
||
|
|
- https://azure.microsoft.com/en-us/pricing/details/ai-document-intelligence/
|
||
|
|
- https://unstructured.io/api-key-hosted#:~:text=Cost%20and%20Usage%20%0AGuidelines
|
||
|
|
|
||
|
|
Здесь evaluation разных Multimodal Large Language Models: https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Evaluation
|
||
|
|
|
||
|
|
**На чем можно писать GUI**
|
||
|
|
- https://dioxuslabs.com/
|
||
|
|
- https://tauri.app
|
||
|
|
|
||
|
|
- какой-то ultra fast tauri + angular setup https://github.com/maximegris/angular-tauri
|