Significant advancements in video diffusion models have brought substantial progress to the field of text-to-video (T2V) synthesis. However, existing T2V synthesis model struggle to accurately generate complex motion dynamics, leading to a reduction in video realism.

Paper page - Searching Priors Makes Text-to-Video Synthesis Better

Check-out the leaderboard here: https://<strong>huggingface</strong>.co/spaces/ArtificialAnalysis/Text-to-Image-Leaderboard

Launching the Artificial Analysis Text to Image Leaderboard & Arena

Transformers have rapidly become the preferred choice for audio classification, surpassing methods based on CNNs. However, Audio Spectrogram Transformers (ASTs) exhibit quadratic scaling due to self-attention. The removal of this quadratic self-attention cost presents an appealing direction.

Paper page - Audio Mamba: Bidirectional State Space Model for Audio Representation Learning

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

stabilityai/stable-audio-open-1.0 · Hugging Face

Discover amazing ML apps made by the community

starline - a Hugging Face Space by mattyamonaca

Stableaudio Open 1.0 - a Hugging Face Space by ameerazam08

Recently video diffusion models have emerged as expressive generative tools for high-quality video content creation readily available to general users. However, these models often do not offer precise control over camera poses for video generation, limiting the expression of cinematic language ...

Paper page - CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generation

We explore uncertainty quantification in large language models (LLMs), with the goal to identify when uncertainty in responses given a query is large. We simultaneously consider both epistemic and aleatoric uncertainties, where the former comes from the lack of knowledge about the ground truth ...

Paper page - To Believe or Not to Believe Your LLM

Text-to-video generation has lagged behind text-to-image synthesis in quality and diversity due to the complexity of spatio-temporal modeling and limited video-text datasets. This paper presents I4VGen, a training-free and plug-and-play video diffusion inference framework, which enhances ...

Paper page - I4VGen: Image as Stepping Stone for Text-to-Video Generation

This work addresses the challenge of video depth estimation, which expects not only per-frame accuracy but, more importantly, cross-frame consistency. Instead of directly developing a depth estimator from scratch, we reformulate the prediction task into a conditional generation problem.

Paper page - Learning Temporally Consistent Video Depth from Video Diffusion Priors

Get the most recent info and news about HuggingFace on HackerNoon, where 10k+ technologists publish stories for 4M+ monthly readers.

HuggingFace

In this article, I’ll introduce Hugging Face — a company that provides tools and platforms for building machine learning applications. We’ll explore what Hugging Face offers, how to use its resources, and how it’s transforming the field of artificial intelligence. Whether you’re an experienced data scientist or a curious beginner, this guide will help you understand and utilize Hugging Face’s capabilities. What is Hugging Face? Let’s start with the basics — what is Hugging Face? Hugging Face initially emerged as a chatbot company that later pivoted to focus on developing cutting-edge open-source NLP technologies. Its flagship library, Transformers, is a game-changer. It simplifies the complex tasks associated with NLP by providing easy access to pre-trained models. This library is built on transformer architectures, celebrated for their ability to handle quantum leaps in processing natural language at scale and with unprecedented accuracy. The beauty of Hugging Face is its democratization of AI technology. By offering accessible tools and models, Hugging Face allows practitioners of various levels to tap into the potential of transformers without needing extensive computational resources or deep expertise in machine learning. How to Get Started with Hugging Face We are going to explore multiple ways to work with Hugging Face. The first way will be through https://huggingface.co/ website. Before you start using it, you must create an account there. https://huggingface.co/ There are three main sections you should know about: Models


Datasets


Spaces Models Models Datasets Datasets Spaces Spaces To use models and datasets, you would need to use the Python language, transformer library, and one of the machine learning frameworks. But if you don’t have programming skills, you can use Spaces to play with different AI models. Hugging Face Models The Hugging Face Model Hub is a repository where you can find pre-trained models for a wide range of tasks, such as natural language processing (NLP), computer vision, audio processing, and more. Hugging Face Model Hub These models are contributed by the community and Hugging Face itself, covering various architectures like BERT, GPT, T5, and others. Users can access thousands of models that have been pre-trained on large datasets, allowing for efficient fine-tuning on their custom tasks. Each model comes with a model card, providing important information such as its intended use case, limitations, and performance metrics. Keep in Mind: Keep in Mind: High-performance models may require significant computational resources and GPU memory to run effectively.
Not all models are free to use for commercial purposes. You should check the specific licensing information provided for each model. High-performance models may require significant computational resources and GPU memory to run effectively. Not all models are free to use for commercial purposes. You should check the specific licensing information provided for each model. Hugging Face Datasets The Datasets library at Hugging Face is designed to provide a simple, efficient way of accessing a wide array of datasets that are useful for machine learning and data-driven projects. Datasets There are datasets for text, audio, image, and tabular data across multiple domains and languages. All of the datasets seamlessly integrate with Hugging Face’s other tools and libraries like Transformers and Tokenizers. Keep in mind: Keep in mind: Some datasets are vast, making them challenging to handle without ample disk space and memory.
Some datasets may have restrictions regarding their use, especially for commercial applications. Check for any licenses before use.
Data might not always be perfect, and additional cleaning or processing may be necessary to fit specific use cases. Some datasets are vast, making them challenging to handle without ample disk space and memory. Some datasets may have restrictions regarding their use, especially for commercial applications. Check for any licenses before use. Data might not always be perfect, and additional cleaning or processing may be necessary to fit specific use cases. Hugging Face Spaces Spaces on Hugging Face are a recent addition, providing an easy-to-use platform for users to deploy their machine learning models and showcase interactive AI applications. Spaces Hugging Face Spaces offers both free and paid options. Free Spaces come with default hardware resources of 16GB RAM, 2 CPU cores, and 50GB of non-persistent disk space. Many models have interactive demos, sharing them with the community without needing your server. You can create public Spaces accessible to all or private ones restricted to selected collaborators or team members. Keep in Mind: Keep in Mind: Restrictions on computational resources may apply, impacting the performance of heavy models or datasets hosted in Spaces.
There might be limitations based on your account level (e.g., free vs. paid subscriptions) that determine the number of Spaces you can maintain and the resources they can consume. Restrictions on computational resources may apply, impacting the performance of heavy models or datasets hosted in Spaces. There might be limitations based on your account level (e.g., free vs. paid subscriptions) that determine the number of Spaces you can maintain and the resources they can consume. How to Use Hugging Face Spaces To explore existing applications on Hugging Face Spaces, follow these steps: 1. Visit the Hugging Face Spaces Directory: Navigate to the Spaces page, where a variety of machine learning applications are showcased. 1. Visit the Hugging Face Spaces Directory: Spaces 2. Browse Applications: On the Spaces page, applications are organized by categories such as Image Generation, Text Generation, Language Translation, and more. 2. Browse Applications: 3. Explore Featured and Trending Spaces: Click on any application name to access its dedicated page, where you can interact with the demo and view additional details. 3. Explore Featured and Trending Spaces: 4. Interact with Applications: Many Spaces offer interactive demos. Follow the on-screen instructions to use the application. 4. Interact with Applications: How to Use Hugging Face Models To use models, we would need to install the transformers library, which offers access to numerous pre-trained models. models, What is Hugging Face Transformers? Transformers are a type of deep learning model architecture that excels at understanding the context and nuances of language. The library provides a plethora of pre-trained models and fine-tuning tools that are invaluable for various tasks such as text classification, tokenization, translation, summarization, and much more. With just a few lines of code, you can integrate these advanced models into your projects, significantly reducing the time and effort typically required to train models from scratch. This accessibility lowers the barrier to entry, fostering a more inclusive environment where more people can innovate with AI. How to Use Hugging Face Transformers Before diving into the specific applications, ensure your development environment is set up correctly. You’ll need it installed on your system: IDE (VS Code or any other)
Python language
Transformers library
Machine Learning Framework (PyTorch or TensorFlow) IDE (VS Code or any other) VS Code Python language Transformers library Machine Learning Framework (PyTorch or TensorFlow) Step 1: Install Necessary Libraries We are going to use the terminal. Use these commands: Install Python: Install Python: sudo apt updatesudo apt install python3 sudo apt updatesudo apt install python3 Use a Virtual Environment Use a Virtual Environment Instead of installing packages globally, create a virtual environment: python3 -m venv venv

source venv/bin/activate python3 -m venv venv

source venv/bin/activate Install Transformers and a few other libraries: Install Transformers and a few other libraries: pip install transformers datasets evaluate accelerate pip install transformers datasets evaluate accelerate You’ll also need to install your preferred machine learning framework. PyTorch and TensorFlow are two of the most popular open-source frameworks for deep learning. PyTorch was developed by Facebook’s AI Research lab (FAIR) and released in 2016. It has gained immense popularity due to its ease of use and flexibility. PyTorch TensorFlow was developed by the Google Brain team and open-sourced in 2015. It’s one of the oldest and most widely adopted frameworks for building deep learning models. TensorFlow Install PyTorch: Install PyTorch: pip install torch pip install torch (Optional) For GPU acceleration, install the appropriate CUDA drivers. Simply follow the instructions on the NVIDIA website https://developer.nvidia.com/cuda-downloads (Optional) https://developer.nvidia.com/cuda-downloads CUDA is a parallel computing platform and application programming interface (API) model created by NVIDIA specifically for its line of GPUs (Graphics Processing Units). It allows developers to utilize NVIDIA GPU hardware for general-purpose processing (GPGPU), extending beyond the traditional use in graphics and enabling significant acceleration of computational tasks in areas like machine learning, scientific computing, and data analysis. Step 2: Explore the Model Hub Navigate to the Hugging Face model hub (https://huggingface.co/models) and explore the available models. https://huggingface.co/models Once you find the one you want to try, click on it and copy the transformer’s code into your IDE (keep in mind that there could be code not only for transformers. For instance, the model https://huggingface.co/Salesforce/blip-image-captioning-base is designed to generate descriptive captions for images. https://huggingface.co/Salesforce/blip-image-captioning-base The code: import requests  
from PIL import Image  
from transformers import BlipProcessor, BlipForConditionalGeneration  
  
processor = BlipProcessor.from_pretrained("Salesforce/blip-image-captioning-base")  
  
model = BlipForConditionalGeneration.from_pretrained("Salesforce/blip-image-captioning-base")  
  
img_url = 'https://storage.googleapis.com/sfr-vision-language-research/BLIP/demo.jpg'   
  
raw_image = Image.open(requests.get(img_url, stream=True).raw).convert('RGB')  
  
# conditional image captioning  
text = "a photography of"  
  
inputs = processor(raw_image, text, return_tensors="pt")  
  
out = model.generate(**inputs)  
  
print(processor.decode(out[0], skip_special_tokens=True))  
  
# >>> a photography of a woman and her dog  
  
# unconditional image captioning  
  
inputs = processor(raw_image, return_tensors="pt")  
  
out = model.generate(**inputs)  
  
print(processor.decode(out[0], skip_special_tokens=True)) import requests  
from PIL import Image  
from transformers import BlipProcessor, BlipForConditionalGeneration  
  
processor = BlipProcessor.from_pretrained("Salesforce/blip-image-captioning-base")  
  
model = BlipForConditionalGeneration.from_pretrained("Salesforce/blip-image-captioning-base")  
  
img_url = 'https://storage.googleapis.com/sfr-vision-language-research/BLIP/demo.jpg'   
  
raw_image = Image.open(requests.get(img_url, stream=True).raw).convert('RGB')  
  
# conditional image captioning  
text = "a photography of"  
  
inputs = processor(raw_image, text, return_tensors="pt")  
  
out = model.generate(**inputs)  
  
print(processor.decode(out[0], skip_special_tokens=True))  
  
# >>> a photography of a woman and her dog  
  
# unconditional image captioning  
  
inputs = processor(raw_image, return_tensors="pt")  
  
out = model.generate(**inputs)  
  
print(processor.decode(out[0], skip_special_tokens=True)) And here is the result: In this way, you can try over 1000 different models. But keep in mind, to effectively run Hugging Face’s Transformers library with PyTorch, it’s recommended to have a system with at least 8 GB of RAM and a GPU with 4 GB of VRAM; however, for optimal performance with larger models, 64 GB of RAM and a GPU with 24 GB of VRAM are advisable. Video Tutorial I also created a detailed video tutorial. Watch it now or save it for later. Watch on YouTube: Hugging Face Tutorial Watch on YouTube: Hugging Face Tutorial Hugging Face Tutorial Conclusion By engaging with Hugging Face’s resources, you can enhance your AI projects and contribute to the broader AI community’s growth and innovation. Please give it a short and share with me your thoughts and learnings below! Cheers! :)

Here's How the Pros Use Hugging Face to Run Countless AI Models on their PCs

NEW YORK, New York, May 7th, 2024/Chainwire/-- Spectral is excited to announce its participation in Hugging Face's Expert Support Program. Spectral is collaborating with deep learning experts from Hugging Face to advance open-source models, datasets, and applications for the Onchain Agent Economy. How To Use Hugging Face Hugging Face plays a critical role across Spectral's organization. Through the Expert Support Program, Spectral regularly meets with the Hugging Face team to plan new use cases, strategize dataset construction, and develop training strategies. Earlier this year, Spectral launched Syntax, which is composed of a fine-tuned LLM orchestrator that routes queries between several tools, including search, Foundry, and an open-source model to generate Solidity code. Building Syntax required custom datasets for both finetuning and RAG, an original Solidity evaluation dataset written by elite smart contract developers, and experimentation across a wide range of open and closed-source models. Spectral's approach to dataset preparation, training, routing, and evaluation has drastically improved during this process, largely due to help from Hugging Face experts. Regular discussions on Spectral's approach mean that the company always receives impartial feedback on its decisions and can reevaluate its strategy as needed. This was important for both the finetuning and evaluation steps, which have now gone through several rounds of improvement thanks to the Hugging Face team. Spectral also relies on Hugging Face for inferences in production through dedicated Inference Endpoints. The fully managed infrastructure allows Spectral to iterate on new models quickly, easily update container and hardware configurations, autoscale with demand, and keep production costs low. Over the coming months, Spectral is excited to continue to open-source its work with onchain datasets and models that interact onchain through Hugging Face. Users can view Spectral's training dataset for the credit scoring challenge and follow its progress here. The Onchain Agent Economy Syntax is pioneering the accessible onchain Agent Economy, inviting users to select agents tailored for their specific tasks. Users can either interact with the foundational agent to generate solidity code or opt for one of the specialized agents, each adept in distinct tasks. For example, we recently launched Syntax MoonMaker, an agent that launches a memecoin project end to end and are soon launching Syntax- TestMachine, an agent that detects vulnerabilities in your smart contracts. Many such agents are currently in production, and upcoming releases of our product will allow users to create their own agents and monetize them on the Syntax network. This system relies on a vast suite of functions available to language models, open-source models for tasks like classification and image generation, fast and reliable blockchain infrastructure, and a trustless system to execute and verify each step. To solve these problems, we've begun developing the Inferchain, optimized to serve the rapidly growing demand for agents and verifiable inferences. Hugging Face Users Joining The Onchain AI Community The importance of open collaboration cannot be overstated. Like many other projects, Spectral would not exist without the work of open-source contributors. Spectral is greatly indebted to everyone who makes this work possible and believes this positive impact should be recognized. As part of this engagement, Spectral is focusing specifically on users powering the AI community. The top individual and small organization contributors on Hugging Face, measured by likes and downloads of their models and datasets, are eligible to register to claim a share of SPEC tokens starting the week of May 6th. Users can check eligibility by signing in with their Hugging Face account here. A snapshot of activity was taken last month, and eligible users will not need an existing wallet to claim. This airdrop is designed to recognize users for their critical work and enable new functionality for the next generation of AI tools. Criteria Spectral reviewed all individual contributors from organizations with less than 10 members that published models or datasets. From these, Spectral took a combined measure of activity from likes and downloads for both models and datasets. Then, Spectral identified the users with the most combined activity from both model and dataset publishers (11298 in both categories) which resulted in a final list of 20,004 unique huggingface users. Spectral is looking forward to engaging further with the open-source community and building the future of AI x web3! https://claims.spectrallabs.xyz/ About Spectral Spectral, a pioneer of the agent economy behind Syntax, is at the forefront of integrating AI with blockchain to democratize development in Web3. Spectral's mission is to simplify the creation and deployment of decentralized applications through autonomous Onchain Agents. Syntax, Spectral’s flagship product, translates natural language into Solidity code, enabling both novices and experts to build on the blockchain effortlessly. With a commitment to transparency and user empowerment, Spectral is shaping a future where anyone can participate in the blockchain revolution. Join Spectral in making this vision a reality and explore the potential of autonomous agents. For more information, visit https://spectrallabs.xyz. For more information, users can follow Spectral on Twitter and Discord. Contact Spectral Labs Team contact@spectral.finance This story was distributed as a release by Chainwire under HackerNoon’s Business Blogging Program. Learn more about the program here. NEW YORK, New York, May 7th, 2024/Chainwire/-- Spectral is excited to announce its participation in Hugging Face's Expert Support Program. Spectral is collaborating with deep learning experts from Hugging Face to advance open-source models, datasets, and applications for the Onchain Agent Economy. How To Use Hugging Face Hugging Face plays a critical role across Spectral's organization. Through the Expert Support Program, Spectral regularly meets with the Hugging Face team to plan new use cases, strategize dataset construction, and develop training strategies. Earlier this year, Spectral launched Syntax, which is composed of a fine-tuned LLM orchestrator that routes queries between several tools, including search, Foundry , and an open-source model to generate Solidity code. Foundry Foundry an open-source model an open-source model Building Syntax required custom datasets for both finetuning and RAG, an original Solidity evaluation dataset written by elite smart contract developers, and experimentation across a wide range of open and closed-source models. Spectral's approach to dataset preparation, training, routing, and evaluation has drastically improved during this process, largely due to help from Hugging Face experts. Regular discussions on Spectral's approach mean that the company always receives impartial feedback on its decisions and can reevaluate its strategy as needed. This was important for both the finetuning and evaluation steps, which have now gone through several rounds of improvement thanks to the Hugging Face team. Spectral also relies on Hugging Face for inferences in production through dedicated Inference Endpoints. The fully managed infrastructure allows Spectral to iterate on new models quickly, easily update container and hardware configurations, autoscale with demand, and keep production costs low. Over the coming months, Spectral is excited to continue to open-source its work with onchain datasets and models that interact onchain through Hugging Face. Users can view Spectral's training dataset for the credit scoring challenge and follow its progress here . here here The Onchain Agent Economy Syntax is pioneering the accessible onchain Agent Economy, inviting users to select agents tailored for their specific tasks. Users can either interact with the foundational agent to generate solidity code or opt for one of the specialized agents, each adept in distinct tasks. For example, we recently launched Syntax MoonMaker, an agent that launches a memecoin project end to end and are soon launching Syntax- TestMachine, an agent that detects vulnerabilities in your smart contracts. Many such agents are currently in production, and upcoming releases of our product will allow users to create their own agents and monetize them on the Syntax network. This system relies on a vast suite of functions available to language models, open-source models for tasks like classification and image generation, fast and reliable blockchain infrastructure, and a trustless system to execute and verify each step. To solve these problems, we've begun developing the Inferchain, optimized to serve the rapidly growing demand for agents and verifiable inferences. Hugging Face Users Joining The Onchain AI Community The importance of open collaboration cannot be overstated. Like many other projects, Spectral would not exist without the work of open-source contributors. Spectral is greatly indebted to everyone who makes this work possible and believes this positive impact should be recognized. As part of this engagement, Spectral is focusing specifically on users powering the AI community. The top individual and small organization contributors on Hugging Face, measured by likes and downloads of their models and datasets, are eligible to register to claim a share of SPEC tokens starting the week of May 6th. Users can check eligibility by signing in with their Hugging Face account here . here here A snapshot of activity was taken last month, and eligible users will not need an existing wallet to claim. This airdrop is designed to recognize users for their critical work and enable new functionality for the next generation of AI tools. Criteria Spectral reviewed all individual contributors from organizations with less than 10 members that published models or datasets. From these, Spectral took a combined measure of activity from likes and downloads for both models and datasets. Then, Spectral identified the users with the most combined activity from both model and dataset publishers (11298 in both categories) which resulted in a final list of 20,004 unique huggingface users. Spectral is looking forward to engaging further with the open-source community and building the future of AI x web3! https://claims.spectrallabs.xyz/ https://claims.spectrallabs.xyz/ https://claims.spectrallabs.xyz/ About Spectral Spectral , a pioneer of the agent economy behind Syntax, is at the forefront of integrating AI with blockchain to democratize development in Web3. Spectral's mission is to simplify the creation and deployment of decentralized applications through autonomous Onchain Agents. Spectral Spectral Syntax, Spectral’s flagship product, translates natural language into Solidity code, enabling both novices and experts to build on the blockchain effortlessly. With a commitment to transparency and user empowerment, Spectral is shaping a future where anyone can participate in the blockchain revolution. Join Spectral in making this vision a reality and explore the potential of autonomous agents. For more information, visit https://spectrallabs.xyz . For more information, users can follow Spectral on Twitter and Discord . https://spectrallabs.xyz Twitter Twitter Discord Discord Contact Spectral Labs Team contact@spectral.finance This story was distributed as a release by Chainwire under HackerNoon’s Business Blogging Program. Learn more about the program here. This story was distributed as a release by Chainwire under HackerNoon’s Business Blogging Program. Learn more about the program here . here here

Spectral Labs Joins Hugging Face’s ESP Program: Advancing The Onchain x Open-Source AI Community

স্পেকট্রাল ল্যাবগুলি আলিঙ্গন মুখের ইএসপি প্রোগ্রামে যোগ দেয়: অনচেইন এক্স ওপেন-সোর্স এআই সম্প্রদায়কে এগিয়ে নিয়ে যাওয়া

Spectral Labs nimmt am ESP-Programm von Hugging Face teil: Förderung der Onchain x Open-Source-KI-Community

Spectral Labs se une al programa ESP de Hugging Face: avanzando en la comunidad de IA de código abierto Onchain x

Spectral Labs rejoint le programme ESP de Hugging Face : faire progresser la communauté d'IA Onchain x Open-Source

स्पेक्ट्रल लैब्स हगिंग फेस के ईएसपी कार्यक्रम में शामिल हुई: ऑनचेन एक्स ओपन-सोर्स एआई समुदाय को आगे बढ़ाना

Spectral Labs が Hugging Face の ESP プログラムに参加: オンチェーン x オープンソース AI コミュニティの発展

Spectral Labs, Hugging Face의 ESP 프로그램에 합류: Onchain x 오픈 소스 AI 커뮤니티 발전

Spectral Labs junta-se ao programa ESP da Hugging Face: Avançando a comunidade Onchain x Open-Source AI

Spectral Labs присоединяется к программе ESP Hugging Face: развитие сообщества искусственного интеллекта с открытым исходным кодом Onchain x

Spectral Labs, Hugging Face'in ESP Programına Katılıyor: Onchain x Açık Kaynak Yapay Zeka Topluluğunu Geliştirme

Spectral Labs tham gia chương trình ESP của Hugging Face: Thúc đẩy cộng đồng AI mã nguồn mở Onchain x

Spectral Labs 加入 Hugging Face 的 ESP 计划：推进 Onchain x 开源 AI 社区

In the rapidly evolving landscape of , the demand for powerful AI solutions has never been higher. Companies and developers are constantly seeking efficient ways to harness the capabilities of AI to build smarter applications, automate tasks, and gain insights from data. Artificial Intelligence ‍ Two platforms, and , have emerged as key players in this domain, each offering unique approaches to meet the diverse needs of the AI community. Eden AI is dedicated to consolidating the finest AI APIs within a single platform, whereas Hugging Face is renowned for its emphasis on open-source models. In this article, we'll explore the differences between these platforms and how they target distinct types of users. Eden AI Hugging Face ‍ Eden AI: Unifying AI APIs Eden AI is a platform designed to simplify the process of accessing and utilizing a wide range of AI APIs. It functions as a centralized hub that aggregates various AI services, making it easier for developers and businesses to experiment with different AI capabilities without having to individually integrate each API. Think of it as a one-stop-shop for AI, offering a seamless experience for accessing text, images, speech processing, and other AI capabilities. ‍ Key Features of Eden AI: Eden AI brings together a diverse set of AI APIs, such as natural language processing (NLP), computer vision, speech recognition, and more, allowing users to access them all from a single platform. Eden AI helps users choose the best AI model at any given time, based on various criteria (accuracy, price, latency, compliance, etc.). API Aggregation: Developers can interact with AI APIs through straightforward and consistent interfaces, reducing the complexity of integration and enabling rapid development. Simplicity: Eden AI is designed to cater to a wide range of users, from individual developers to large enterprises, providing scalable solutions to meet varying demands. Scalability: Users can control their AI usage and costs by managing their API subscriptions within one platform, streamlining billing and payment processes. Cost-Efficiency: ‍ Hugging Face: The Open Source Model Hub , on the other hand, is renowned for its focus on Open Source models. It provides a repository of pre-trained models and tools for natural language processing tasks, making state-of-the-art AI models accessible to developers and researchers worldwide. The core philosophy of Hugging Face is built around the idea of collaboration and open sharing within the AI community. Hugging Face ‍ Key Features of Hugging Face : Hugging Face offers a vast library of pre-trained AI models for a wide range of NLP tasks, such as text classification, language translation, and text generation. Open Source Models The platform encourages contributions from the AI community, allowing developers to fine-tune and share their own models, democratizing AI model development. Community-Driven: Hugging Face provides tools and resources for fine-tuning pre-trained models on specific tasks or domains, enabling customization for various use cases. Fine-Tuning: It serves as a valuable resource for researchers, students, and developers who want to experiment with and learn about cutting-edge AI models and techniques. Research and Education: ‍ Targeting Different User Needs While both Eden AI and Hugging Face cater to the AI community, they serve different purposes and target distinct types of users: ‍ Primarily targets businesses, developers, and organizations looking for quick and efficient access to a wide array of AI capabilities. It's ideal for those who want to leverage AI APIs without the need to delve into model development intricacies. Eden AI simplifies integration, reduces development time, and offers cost-effective solutions. Eden AI: Appeals to users with in-depth knowledge of AI, including researchers, data scientists, and developers. It excels in providing open access to state-of-the-art AI models, fostering collaboration, and enabling fine-tuning for specific tasks. Hugging Face is the go-to platform for those interested in model research, experimentation, and customization. Hugging Face: ‍ Conclusion Both Eden AI and Hugging Face are pivotal players in the AI ecosystem, each with a distinct focus and set of offerings. Eden AI simplifies AI integration by aggregating diverse AI APIs, while Hugging Face empowers the AI community with open access to pre-trained models and tools for customization. The choice between these platforms depends on the specific needs and goals of users, whether it's quick AI integration or deep model exploration and development. Together, they contribute to the democratization and advancement of AI technology. Also published here.

Eden AI vs Hugging Face: Use Cases,  Target Users and Value Propositions 

In this blog, I will show you how to quickly create an entirely free app using a JavaScript program that runs within the free Oracle database and calls AI, the results of which are stored in the database and can then be accessed using , , Rest, or any other language. All of the source code is, of course, available . Hugging Face SQL JSON here A typical sequence of actions in applications, both in general and particularly in AI applications, involves an application utilizing an AI service. The acquired information is then stored in a database, ready for retrieval, subsequent processing (such as using OML), analysis, and future queries. When the command originates directly from the database, the data is promptly stored and gains higher reliability. This approach is in-process and eliminates the need for an additional network call for persistence. Additionally, by housing the logic and even code within the database, the remarkable features of the robust Oracle database, including high availability, management, and observability, are naturally harnessed. This distinct capability is a hallmark of the Oracle database, which encompasses both Java and, as detailed in this blog, JavaScript runtime engines within the database itself. Hugging Face specializes in crafting tools for constructing applications using machine learning. Particularly notable is its transformers library designed for natural language processing applications, along with its platform facilitating the sharing of machine learning models and datasets. It has garnered significant popularity in recent years. In short, we’ll… Create a cohere.com or huggingface.co account, create and copy an access token, and select a model. Create the Oracle database. Create the database user and table(s). Run a short JavaScript program in the database to make a call to the Cohere or Hugging Face AI model and store the results in the table. Query these results using SQL, JSON, Rest, etc. Setup Cohere account and AI model Go to https://cohere.com and , go to your , and click the button. Sign Up Profile Settings Click on , then API Keys, create a key (or use trial) and copy its value for use later. Dashboard Click on and select the hamburger menu/icon on the left of any page to view the API Reference list and select one. In this case, we will select section and API Reference generate Co.Generate. Select and then select the option in order to see a sample code snippet to call the model. Ianguage JavaScript Setup Hugging Face account and AI model Go to https://huggingface.co and , go to your , and click the button. Sign Up Profile Settings Click on , create a token, and copy its value for use later. Access Tokens Click on and select a model. In this case, we will select a model under the section for Models Natural Language Processing Question Answering. Select a model (perhaps one of the more popular ones), notice the information in the on the left, and then select the drop-down menu on the right. Model Card Deploy Select and then select the option in order to see a sample code snippet to call the model. Inference API JavaScript Setup Oracle Database We can use any flavor of the Oracle database 23c. On the cloud, we can use Oracle always free autonomous database later this year when 23c should be available. We can use the Oracle 23c Free version as it’s available now and we can simply install or use a container image locally. Or of course, we could use the local for dev and cloud for production, etc. and either of these options is very quick to set up… Oracle Database Free… You can go here to set up the Free Oracle Database. Using the container image is very simple. You can simply pull the image: docker pull container-registry.oracle.com/database/free:latest Then, execute the one line below, replacing with a password of your choice (you can also add a parameter with a directory location in order to persist changes across restarts). Notice the param that allows calls out of the container. This is the setting for Macs and will be different if running on a different OS. -e ORACLE_PASSWORD=Welcome12345 -v oracle-volume:/somedirectory --add-host docker.for.mac.host.internal:host-gateway docker run -d -p 1521:1521 — add-host docker.for.mac.host.internal:host-gateway -e ORACLE_PASSWORD=Welcome12345 gvenzl/oracle-free Setup SQLcl (or Database Actions) and connect You can install SQLcl to manage the database by: Downloading and install from this location https://www.oracle.com/database/sqldeveloper/technologies/sqlcl/download/ This will provide an executable that we will use to administer the database. For convenience, you may add to your PATH. [SQLcl_INSTALL_DIR]/bin/sql [SQLcl_INSTALL_DIR]/bin Login replacing with the location of your SQLcl and replacing with the one you provided as when creating the database Here is an example when using a local install (eg Oracle Database Free container image): [SQLcl_INSTALL_DIR] Welcome12345 ORACLE_PASSWORD [SQLcl_INSTALL_DIR]/bin/sql /nolog SQL> connect sys@//localhost:1521/freepdb1 as sysdba Password? (**********?) ************* Connected. Create the users, with appropriate privileges, that will call Cohere and/or Hugging Face via JavaScript. Run the following SQL files to create the user and its necessary grants and ACLs (note that these can be further tightened for better security). aijs @sql/create_aijs_acl.sql SQL> @sql/create_aijs_user.sql``SQL> Now connect the user and create a table to store the results of the call to Hugging Face SQL> connect aijs/Welcome12345; SQL> create table huggingfacejson (id json); SQL> create table coherejson (id json); Now everything is ready to run! Run either the Cohere or HuggingFace queries from the JavaScript code in the database Cohere Example SQL> @sql/coherequery.sql PL/SQL procedure successfully completed. Then, check the JSON results stored in the table by doing a SQL query. SQL> select * from coherejson; [{"id":"1b42f8f9-ea6d-4a65-8f02-bdcf89d9bd79","generations":[{"id":"d6467c0b-4a78-4dd4-b822-c1e2bfa4ecb0","text":"\n LLMs or Large Language Models are artificial intelligence tools that can read, summarize and translate texts and"}],"prompt":"Please explain to me how LLMs work","meta":{"api_version":{"version":"1"}}} Looking at the code we just executed we can see the JavaScript module created and the function being called: create or replace mle module cohere_module language javascript as import "mle-js-fetch"; import "mle-js-oracledb"; export async function cohereDemo(apiToken) { if (apiToken === undefined) { throw Error("must provide an API token"); } const modelId = "generate"; const restAPI = `https://api.cohere.ai/v1/${modelId}`; const headers = { accept: 'application/json', 'content-type': 'application/json', "Authorization": `Bearer ${apiToken}` }; const payload = { max_tokens: 20, return_likelihoods: 'NONE', truncate: 'END', prompt: 'Please explain to me how LLMs work' }; const resp = await fetch( restAPI, { method: "POST", headers: headers, body: JSON.stringify(payload), credentials: "include" } ); const resp_json = await resp.json(); session.execute( `INSERT INTO COHEREJSON (id) VALUES (:resp_json)`, [ JSON.stringify(resp_json) ] ); } / create or replace procedure coherequery( p_API_token varchar2 ) as mle module cohere_module signature 'cohereDemo(string)'; / -- this is how you can test the API call begin utl_http.set_wallet('system:'); coherequery('[yourcoheretokenhere]'); end; / Hugging Face Example SQL> @sql/huggingfacequery.sql PL/SQL procedure successfully completed. Then, check the JSON results stored in the table by doing a SQL query. SQL> select * from huggingfacejson; [{"score":0.16964051127433777,"token":2053,"token_str":"no","sequence":"the answer to the universe is no."},{"score":0.07344779372215271,"token":2498,"token_str":"nothing","sequence":"the answer to the universe is nothing."},{"score":0.05803246051073074,"token":2748,"token_str":"yes","sequence":"the answer to the universe is yes."},{"score":0.043957971036434174,"token":4242,"token_str":"unknown","sequence":"the answer to the universe is unknown."},{"score":0.04015727713704109,"token":3722,"token_str":"simple","sequence":"the answer to the universe is simple."}] Looking at the code we just executed we can see the JavaScript module created and the function being called: create or replace mle module huggingface_module language javascript as import "mle-js-fetch"; import "mle-js-oracledb"; export async function huggingfaceDemo(apiToken) { if (apiToken === undefined) { throw Error("must provide an API token"); } const payload = { inputs: "The answer to the universe is [MASK]." }; const modelId = "bert-base-uncased"; const headers = { "Authorization": `Bearer ${apiToken}` }; const restAPI = `https://api-inference.huggingface.co/models/${modelId}`; const resp = await fetch( restAPI, { method: "POST", headers: headers, body: JSON.stringify(payload), credentials: "include" } ); const resp_json = await resp.json(); session.execute( `INSERT INTO HUGGINGFACEJSON (id) VALUES (:resp_json)`, [ JSON.stringify(resp_json) ] ); } / create or replace procedure huggingfacequery( p_API_token varchar2 ) as mle module huggingface_module signature 'huggingfaceDemo(string)'; / -- this is how you can test the API call begin utl_http.set_wallet('system:'); huggingfacequery('[hf_yourhuggingfacetokenhere]'); end; / Useful parameters and debug information for Hugging Face can be found . here The results can now be queried, analyzed, etc. using SQL or JSON (simultaneously thanks to the new JSON duality feature), REST, or even the MongoDB API.Oracle Database is the perfect database for AI for a number of reasons, particularly as it is a vector database with capabilities such as the following already available today: Native data types for vector representation and storage: RAW, BLOB, JSON In-Memory Column Store to store & search vector embeddings w/ SIMD kernels for amazing performance Extensible indexing framework to create data-model specific indices (e.g., Text, Spatial) Native Oracle Machine Learning APIs — Modeling, Classification, Scoring, Clustering, etc. Plus, many other capabilities which, used together, make Oracle a better Vector Database DMLs, Parallel Loading, Partitioning, Advanced Compression, Parallel Query, RAC, Sharding, etc. It is also possible to call Oracle OCI AI and other services from within the Oracle database. Conclusion The blog went into how to call Hugging Face APIs from within the Oracle database using JavaScript thus demonstrating a powerful combination of features perfectly suited to a broad array of AI solutions and friendly to JavaScript developers. Some other blogs related to JavaScript in the database, in general, the MultiLingual Engine (MLE) that makes it possible, etc. can be found and . Thanks to Martin and Lucas for all their help. in Martin Bach’s blogs this post about JavaScript modules I look forward to any comments or questions you may have and really appreciate your time reading. Also published . here

Calling Hugging Face AI From an Oracle Database Using JavaScript

Hugging Face offers solutions and tools for developers and researchers. With an open-source library called Transformers, they provide access to state-of-the-art pre-trained models. Due to their high quality and readily available datasets covering a diverse number of topics and languages, Hugging Face has contributed to the advancement of NLP technology. This article looks at the accessible to developers and researchers worldwide. Best Hugging Face Datasets for Building NLP Models, Ultimate List of Hugging Face Datasets 1. IMDB The IMDB dataset is commonly used for natural language processing tasks, specifically sentiment analysis. It consists of a collection of 50,000 movie reviews from the Internet Movie Database website, split evenly between positive and negative reviews. The dataset is used for binary classification, which means the main goal is to classify each review as either positive or negative based on the sentiment expressed in the text. Each review in this dataset is a text document, and the dataset has been preprocessed and cleaned so it’s ready to use for training and evaluating sentiment analysis models. The IMDB dataset is available on the Hugging Face platform, making it easy to access and use with popular NLP libraries like the Hugging Face Transformers library. Note: Due to the fact that this dataset only includes movie reviews in English and may not represent other types of text, it is limited. 2. Common Voice This dataset is a large, open-source dataset of human voice recordings created by Mozilla for use in developing and training speech recognition models. It is unique because it is designed to be multilingual, with recordings in over 60 different languages. The Common Voice Dataset is one of the largest publicly available datasets for speech recognition because it comprises over 9,000 hours of speech data from over 60,000 contributors worldwide. Each recording in the dataset corresponds with metadata, including the gender, speaker’s age, accent and text of the spoken sentence. It is available in a variety of formats, like FLAC, MP3, and WAV. Each recording in the dataset is associated with metadata, including the speaker's age, gender, and accent, as well as the text of the spoken sentence. The dataset is available on Hugging Face as part of the common voice package in a variety of formats, including FLAC, MP3, and WAV, making it easy to work with in a variety of software environments. 3. Amazon Polarity The Amazon Polarity dataset is a collection of text reviews and associated star ratings for products sold on Amazon. The dataset consists of over 3 million reviews in multiple languages, including English, German, Japanese, and French, which covers a wide range of product categories, including books, electronics and clothing. The goal of the dataset is to offer a large-scale benchmark for sentiment analysis, where the task is to predict the sentiment of a given review as positive or negative based on the associated star rating. This dataset is pre-processed, labelled and available on Hugging Face as part of the amazon_polarity package, which provides a comfortable interface for loading and working with the dataset in Python. 4. Silicone This dataset comprises classifications of sentences based on types: directive, commissive, informative or interrogative. These sentences are gathered from various domains, such as television dialogue, phone conversations, etc. The Silicone dataset has all data points in English. With insights into how spoken language systems can be designed to handle different sentence types across various domains, it is also a valuable resource for training and evaluating natural language models. 5. Yahoo Answers Topics The Yahoo Answers Topics dataset is a collection of over 1 million questions and their corresponding topics, crawled from Yahoo Answers. The dataset includes a wide variety of subjects, including sports, business & finance, society & culture, science & mathematics, family & relationships, computers & the internet, and more. Each question in the dataset is labeled with one or more topic categories, making it useful for developing models that can accurately classify text data into multiple categories. The dataset is preprocessed and tokenized, which makes it easy to train a model to categorize certain questions and answers into one of these categories. 6. Emotion The Emotion dataset is a collection of texts labeled with emotion categories. The dataset includes texts that express six primary emotions: anger, fear, joy, love, sadness, and surprise. The texts in this dataset are sourced from a variety of public datasets and online resources, such as Reddit, Twitter and news articles. Each text is labeled with one of the six primary emotion categories, allowing the dataset to be used for training and evaluating natural language processing models that can classify text data into these emotion categories. One special feature of the Emotions dataset is its focus on a diverse range of emotions, which allows for the training of models that can recognize and distinguish between a variety of emotional states. This is useful for building models for applications such as sentiment analysis or chatbots. 7. Hate Speech18 This dataset is a collection of large text data with over 24,000 labeled instances, which can be useful for training deep learning models. It is labeled as hate speech or not hate speech and includes a variety of text types, such as tweets, Facebook posts and YouTube comments. The Hate Speech dataset is labeled with three categories: hate speech offensive language neither The "hate speech category" consists of texts that express hatred or prejudice towards a particular group based on their race, gender, religion, or other characteristics. The "offensive language" category includes texts that may not express direct hatred but are still considered highly offensive or inappropriate. One special feature of the Hate Speech Dataset is its specialization on hate speech and offensive language, which is necessary for building models that can recognize and combat online harassment and discrimination. 8. SMS Spam The SMS Spam dataset contains over 5000 SMS messages classified as spam or ham (non-spam). It is collected from real users' mobile phones and is commonly used in machine learning and natural language processing research for training models that can distinguish between spam and legitimate messages. This can be useful for developing spam filters or identifying potential fraudulent messages. 9. Banking 77 It is a comprehensive and intricate collection of financial text data that includes over 77,000 documents. Among these documents are over 13,000 customer messages that were sent to banks, detailing various complaints and issues. These customer messages are categorized into seventy-seven different intents, including inquiries about card arrival, card malfunctions, extra charges on the card, and declined transfer issues. By using the Banking 77 dataset, banks can improve their response times and better manage customer concerns. Additionally, other businesses that receive high volumes of customer requests could use the Banking 77 dataset as a model to build similar tools to categorize and analyze customer messages. However, it's important to note that preparing the dataset for machine learning requires extensive filtering and processing to ensure accuracy and quality. 10. Scan The Scan dataset comprises over 7,000 questions that cover a diverse range of topics, such as sports, science, and entertainment, and is a large-scale benchmark for evaluating natural language understanding (NLU) models. It is designed to test a model's ability to perform various types of reasoning, including temporal reasoning, spatial reasoning, and logical reasoning. Each question is paired with a set of possible answers, and the goal of the model is to select the correct answer from the options provided. Common Use Cases for Hugging Face Datasets - The IMDB dataset can be used for sentiment analysis and text classification tasks, such as determining whether a movie review is positive or negative. IMDB - The Common Voice dataset is useful for speech recognition tasks, such as building speech-to-text models for different languages. Common Voice - The Amazon Polarity dataset can be used for sentiment analysis and text classification tasks, such as determining whether a product review on Amazon is positive or negative. Amazon Polarity - The Silicone dataset is designed for text classification tasks, specifically identifying the type of sentence as commissive, directive, informative, or a question. Silicone - The Yahoo Answers Topics dataset can be used for topic classification tasks, such as determining the topic or subject matter of a given text. Yahoo Answers Topics - The Emotion dataset can be used for emotion classification tasks, such as determining the emotion or sentiment expressed in a text. Emotion - The Hate Speech dataset can be used for detecting and analyzing hate speech, which is useful for identifying and combating harmful language online. 18 Hate Speech - The SMS Spam dataset can be used for detecting spam text messages, which is useful for filtering unwanted messages in SMS inboxes. SMS Spam - The Banking 77 dataset can be used for intent classification tasks, such as categorizing customer messages to banks into different intent categories, such as inquiries about card arrival, issues with card not working, or extra charges on a card. Banking 77 - The Scan dataset can be used for text classification tasks, such as identifying the type of text as being news or academic and can be used for training models for tasks such as text summarization and text generation. Scan Final Thoughts Hugging Face has a broad range of datasets available that cater to different natural language processing tasks, from text classification to language modeling. With Hugging Face's growing collection of datasets and consistent updates, it has become a popular and reliable source for NLP data. These datasets are available for anyone to download and use freely. More Dataset Listicles: Power Bi Datasets Time Series Datasets Geospatial Datasets

10 Best Hugging Face Datasets for Building NLP Models

The from hugging Face provides a very efficient way to load and process NLP datasets from raw files or in-memory data. These NLP datasets have been shared by different research and practitioner communities across the world. Datasets library You can also load various evaluation metrics used to check the performance of on numerous tasks. NLP models If you are working in and want an NLP dataset for your next project, I recommend you use this library from Hugging Face. Natural Language Processing You can use this library with other popular frameworks in machine learning, such as , , , and You will learn more in the examples below. Numpy Pandas Pytorch TensorFlow. The NLP datasets are available on different tasks such as Text-classification Question-Answering Sequence-modelling Machine translation, Automatic speech-recognition Sentiment Analysis Speech Processing Name entity Recognition Part of Speech Tagging others You can find the NLP datasets in more than 186 languages. All these datasets can also be browsed on the and can be viewed and explored online with the . HuggingFace Hub Datasets viewer In this article, you will learn how to download, load, configure and use NLP datasets from the hugging face datasets library. Let's get started. How to Install Datasets Library Installation is easy and takes only a few minutes. You can use pip as follows: pip install datasets Another option for installation is using conda as follows. conda install -c huggingface -c conda-forge datasets List of Datasets To view the list of different available datasets from the library, you can use the function from the library. list_datasets() datasets list_datasets, load_dataset pprint pprint

datasets_list = list_datasets() 
pprint(datasets_list,compact= ) from import from import True Currently, the datasets library has a total of 1182 datasets that can be used to create different NLP solutions. List of Dataset with Details You can also view a list of the datasets with details by adding an argument called equal to in the list_datases() function as follows. with_details True datasets_list = list_datasets(with_details= ) 
pprint(datasets_list) True This will show a list of datasets with their id(name), descriptions, and files (if downloaded in your local machine). Load Dataset To load the dataset from the library, you need to pass the file name on the function. load_dataset() The load_dataset function will do the following. Download and import in the library the file processing script from the Hugging Face GitHub repo. Run the file script to download the dataset Return the dataset as asked by the user. By default, it returns the entire dataset dataset = load_dataset( , ) 'ethos' 'binary' In the above example, I downloaded the dataset from hugging face. ethos "ETHOS: onlinE haTe speecH detectiOn dataSet. This repository contains a dataset for hate speech detection on social media platforms, called Ethos. There are two variations of the dataset:"- HuggingFace's page Each dataset can have several configurations that define the sub-part of the dataset you can select. For example, the ethos dataset has two configurations. Note: binary version multi-label version Let’s load the binary version we have downloaded. print(dataset) Split the Train and Validation Sets Sometimes you don't want to download the entire dataset, you can use the split parameter to specify if you want to download the train, validation, or test set from the dataset. ethos_train = load_dataset( , ,split= )

ethos_validation = load_dataset( , ,split= ) 'ethos' 'binary' 'train' 'ethos' 'binary' 'validation' This will save the training set in the variable and the validation set in variable. ethos_train ethos_validation Not all datasets have the train, validation, and test set. Some of them can contain only the train set. So you need to read more about the dataset you want to download from the . Note: hugging face dataset page Load Dataset from Local File You can use the library to load your local dataset from the local machine. You can load datasets that have the following format. CSV files JSON files Text files (read as a line-by-line dataset), Pandas pickled dataframe To load the local file you need to define the format of your dataset (example "CSV") and the path to the local file. dataset = load_dataset( , data_files= ) 'csv' 'my_file.csv' You can similarly instantiate a Dataset object from a pandas DataFrame as follows: datasets Dataset pandas pd 

df = pd.DataFrame({ : [ , , ]})
df_dataset = Dataset.from_pandas(df)

print(df_dataset) from import import as "a" 1 2 3 Download Mode When you load already downloaded data from the cache directory, you can control how the function handles it by setting its download_mode parameter. load_datasets() The parameter options are as follows. (default) - This will reuse both raw downloads and the prepared dataset if they exist in the cache directory. "reuse_dataset_if_exists" This will use the cached dataset. "reuse_cache_if_exists"- - This will redownload the dataset. "force_redownload" dataset = load_dataset( , , download_mode= ) 'ethos' 'binary' "force_redownload" In the above example, we redownload the ethos dataset with binary configuration. Setting The Format You can set the format of the dataset instance by using the function that takes as arguments. set_format() an optional string that defines the type of the objects that should be returned by : type: datasets.Dataset.getitem() (default): return python objects, None/'python' return PyTorch tensors, 'torch'/'pytorch'/'pt': : return Tensorflow tensors, 'tensorflow'/'tf' return JAX arrays, 'jax': : return Numpy arrays, 'numpy'/'np' : return Pandas DataFrames 'pandas'/'pd' ethos_train.set_format(type= , columns=[ , ]) 'pandas' 'comment' 'label' In the above example, we set the format type as . "pandas" Final Thoughts on NLP Datasets from Huggingface In this article, you have learned how to download datasets from hugging face datasets library, split into train and validation sets, change the format of the dataset, and more. We did not cover all the functions available from the datasets library. Check the following resources if you are looking to go deeper. Datasets: https://huggingface.co/datasets Datasets documentation: https://huggingface.co/docs/datasets/ NLP Metrics: https://huggingface.co/docs/datasets/loading_metrics.html If you learned something new or enjoyed reading this article, please share it so that others can see it. Until then, see you in the next post! You can also find me on Twitter . @Davis_McDavid And you can read more articles like this . here Want to keep up to date with all the latest in Data Science? in the footer below. Subscribe to our newsletter

NLP Datasets from HuggingFace: How to Access and Train Them

Conjuntos de datos de PNL de HuggingFace: cómo acceder a ellos y entrenarlos

Photo by on Mick De Paola Unsplash The “Maybe just a quick one” series title is inspired by my most common reply to “Fancy a drink?”, which, may or may not end up in a long night. Likewise, these posts are intended to be short but I get carried away sometimes, so, apologies in advance. About 🤗 Transformers 🤗 Transformers ( ) is a collection of state-of-the-art NLU (Natural Language Understanding) and NLG (Natural Language Generation ) models.  They offer a wide variety of architectures to choose from (BERT, GPT-2, RoBERTa etc) as well as a of pre-trained models uploaded by users and organisations. Hugging Face transformers hub Fine-tuning a model One of the things that makes this library such a powerful tool is that we can use the models as a basis for tasks. In other words, they can be a starting point to apply some fine-tuning using our own data. The library is designed to easily work with both Tensorflow or PyTorch. transfer learning 🤗 Datasets is a  wrapper library that provides some tools to load and process data in many commonly used formats (CSV, JSON etc). It also makes sharing datasets and metrics for Natural Language Processing extremely easy. Hugging Face Datasets 🤗 Datasets originated from a fork of the awesome TensorFlow Datasets and the HuggingFace team want to deeply thank the TensorFlow Datasets team for building this amazing library. Well, let’s write some code In this example, we will start with  a pre-trained model and fine-tune it on the dataset. We will then test it  on classifying tweets as hate speech, offensive language, or neither. All coding is done in . BERT (uncased) Hate Speech and Offensive Language Google Colab Please note:  this dataset contains text that can be considered racist, sexist, homophobic, or generally offensive. So let’s start by installing some necessary packages, import them and load the dataset. The dataset is stored in Google Drive, and the path to load it from is . So if you code along , please make sure you change the path to point to your own dataset file. /content/drive/MyDrive/Data/labeled_data.csv We are using the function to load it and then split it into train, validation, and test sets. The 3 sets are then gathered together to form a .This is a dictionary class, that offers us many methods to process the data. We will then remove some of the columns we don’t need for our classification task. load_dataset DatasetDict !pip install transformers
!pip install datasets datasets load_dataset,DatasetDict transformers AutoTokenizer,TFAutoModelForSequenceClassification tensorflow tf numpy np matplotlib.pyplot plt

The dataset located our Google Drive data folder.

DATA_PATH = dataset = load_dataset( , data_files=DATA_PATH,split= )
train_testvalid = dataset.train_test_split()
test_valid = train_testvalid[ ].train_test_split()
train_test_valid_dataset = DatasetDict({ : train_testvalid[ ], : test_valid[ ], : test_valid[ ]})
dataset = train_test_valid_dataset.remove_columns([ , , , , ]) #Install the necessary packages from import from import import as import as import as is in "/content/drive/MyDrive/Data/labeled_data.csv" 'csv' 'train' 'test' 'train' 'train' 'test' 'test' 'valid' 'train' 'hate_speech' 'offensive_language' 'neither' 'Unnamed: 0' 'count' So now we need to preprocess the data. The tool responsible for this is a What do tokenizers do? Very simply put, they split the data in tokens (these can be characters, words, part of words, depending on the model), and convert them into tensors of numeric ids, which is the form that the model can read. For this task, we are using the tokenizer from the pre-trained model we selected ( ). But let’s see how we achieve this: Tokenizer. bert-base-cased tokenizer = AutoTokenizer.from_pretrained( ) tokenizer(examples[ ], padding= , truncation= )

tokenized_datasets = dataset.map(tokenize_function, batched= )
train_dataset = tokenized_datasets[ ]
eval_dataset = tokenized_datasets[ ]
test_dataset = tokenized_datasets[ ]
tf_train_dataset = train_dataset.remove_columns([ ]).with_format( )
tf_eval_dataset = eval_dataset.remove_columns([ ]).with_format( )
tf_test_dataset = test_dataset.remove_columns([ ]).with_format( )

train_features = {x: tf_train_dataset[x].to_tensor() x tokenizer.model_input_names}
train_tf_dataset = tf.data.Dataset.from_tensor_slices((train_features, tf_train_dataset[ ]))
train_tf_dataset = train_tf_dataset.shuffle(len(tf_train_dataset)).batch( )

eval_features = {x: tf_eval_dataset[x].to_tensor() x tokenizer.model_input_names}
eval_tf_dataset = tf.data.Dataset.from_tensor_slices((eval_features, tf_eval_dataset[ ]))
eval_tf_dataset = eval_tf_dataset.batch( )

test_features = {x: tf_test_dataset[x].to_tensor() x tokenizer.model_input_names}
test_tf_dataset = tf.data.Dataset.from_tensor_slices((test_features, tf_test_dataset[ ]))
test_tf_dataset =test_tf_dataset.batch( ) "bert-base-cased" : def tokenize_function (examples) return "tweet" "max_length" True True "train" "valid" 'test' "tweet" "tensorflow" "tweet" "tensorflow" "tweet" "tensorflow" for in "class" 8 for in "class" 8 for in "class" 8 Notice how we used the dataset function, to apply our user-defined to all the elements of the dataset. map tokenize_function After applying the tokenization, we created 3 to feed the model with. We are now ready to train the model. Again, we are using the selected pre-trained model, transferring the “knowledge” it already has, but replacing its head with one that is suited to our task. We are using the class, which represents a generic Tensorflow (hence the TF prefix) model, with a sequence classification head. Also, notice the num_labels parameter which is set to 3, as this is a multi-class task with 3 distinct labels. After the training is finished, we plot the Sparse Categorical Accuracy and the Loss of both the train and the validation dataset. Tensorflow Datasets TFAutoModelForSequenceClassification model = TFAutoModelForSequenceClassification.from_pretrained( , num_labels= )
model.compile(
    optimizer=tf.keras.optimizers.Adam(learning_rate= ),
    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits= ),
    metrics=tf.metrics.SparseCategoricalAccuracy(),
)

history = model.fit(train_tf_dataset, validation_data=eval_tf_dataset, epochs= )

plt.plot(history.history[ ])
plt.plot(history.history[ ])
plt.title( )
plt.ylabel( )
plt.xlabel( )
plt.legend([ , ], loc= )
plt.show()
plt.plot(history.history[ ])
plt.plot(history.history[ ])
plt.title( )
plt.ylabel( )
plt.xlabel( )
plt.legend([ , ], loc= )
plt.show() "bert-base-cased" 3 5e-5 True 2 'sparse_categorical_accuracy' 'val_sparse_categorical_accuracy' 'model sparse categorical accuracy' 'accuracy' 'epoch' 'train' 'val' 'upper left' 'loss' 'val_loss' 'model loss' 'loss' 'epoch' 'train' 'val' 'upper left' We can now evaluate the model on the test dataset we created earlier: test_loss, test_acc = model.evaluate(test_tf_dataset,verbose= )
print( , test_acc) / - s - loss: - sparse_categorical_accuracy: Test accuracy: model.save_pretrained( ) 2 '\nTest accuracy:' 194 194 62 0.2596 0.9135 0.9134925603866577 "/content/drive/MyDrive/Data/hate-speech-bert" Notice that we save the model with the function offered by Transformers. This action generates a directory with two files by default: a .json file that contains the model configuration and a .h5 file with the model weights. We can also push the model to the Hugging Face Models Hub should we want to, in order to make it available to the public. save_pretrained Does it work though? Let’s see how our model does in classifying some unseen text. I will use some stereotypical racist/offensive/sexist texts posted on social media. : Warning Due to the nature of this task, the language used here can be racist, sexist, and offensive. However, this is the only way to evaluate the model’s ability. pred2label = { : , : , : }
preds = model(tokenizer([ , , ],return_tensors= ,padding=True,truncation=True))[ ]
print(preds)
class_preds = np.argmax(preds, axis= ) pred class_preds:
  print(pred2label[pred]) 0 'Hate Speech' 1 'Offensive Language' 2 'Neither' "Jews are useless , I don't see why they even exist" "Gay people suck" "Women are dressed up like whores these days" "tf" 'logits' 1 for in tf.Tensor(
[[ ]
 [ ]
 [ ]], shape=( , ), dtype=float32)
Hate Speech
Hate Speech
Offensive Language 0.37532297 0.14053927 -0.8647832 0.04699412 -0.17951615 -0.3738104 0.2524849 2.586107 -2.8212454 3 3 Looks like our model managed to correctly (well that’s subjective, but generally speaking these look correct) classify the texts. Further reading: If the above seems interesting to you, there is a lot more that this library can do. I would start by checking their documentation which is quite extensive as well as some quick video courses they provide: 🤗 Transformers Model Hub 🤗 Datasets (with Youtube Videos, we always like them ) Crash course Fine-tuning with custom datasets Happy coding.

How to Fine Tune a 🤗 (Hugging Face) Transformer Model

Transformer-based models are a game-changer when it comes to using unstructured text data. As of September 2020, the top-performing models in the General Language Understanding Evaluation (GLUE) benchmark are all BERT transformer-based models. At , we often encounter scenarios where we have supporting tabular feature information and unstructured text data. We found that by using the tabular data in these models, we could further improve performance, so we set out to build a toolkit that makes it easier for others to do the same. Georgian Building on Top of Transformers The main benefits of using transformers are that they can learn long-range dependencies between text and can be trained in parallel (as opposed to sequence to sequence models), meaning they can be pre-trained on large amounts of data. Given these advantages, BERT is now a staple model in many real-world applications. Likewise, with libraries such as , it’s easy to build high-performance transformer models on common NLP problems. HuggingFace Transformers Currently, transformer models using unstructured text data are well understood. However, in the real-world, text data is often supported by rich structured data or other unstructured data like audio or visual information. Each one of these might provide signals that one alone would not. We call these different ways of experiencing data—audio, visual or text—modalities. Think about E-commerce reviews as an example. Beyond the review text itself, information about the seller, buyer and product is available as numerical and categorical features. We set out to explore how text and tabular data could be used together to provide stronger signals in our projects. We started by exploring the field known as multimodal learning, which focuses on how to process different modalities in machine learning. Multimodal Literature Review The current models for multimodal learning mainly focus on learning from the sensory modalities such as audio, visual, and text. Within multimodal learning, there are several branches of research. The MultiComp Lab at Carnegie Mellon University provides an excellent . Our problem falls under Multimodal Fusion—joining information from two or more modalities to make a prediction. taxonomy As text data is our primary modality, we focus on the literature that treats text as the main modality and introduces models that leverage the transformer architecture. Trivial Solution to Structured Data Before we dive into the literature, there is a simple solution that can be used where the structured data is treated as regular text and is appended to the standard text inputs. Taking the E-commerce reviews example, the input can be structured as follows: Review. Buyer Info. Seller Info. Numbers/Labels. Etc. One caveat with this approach, however, is that it is limited by the maximum token length that a transformer can handle. Transformer on Images and Text In the last couple of years, transformer extensions for image and text have advanced significantly. by Kiela et al. (2019) uses pre-trained ResNet and pre-trained BERT features on unimodal images and text respectively and feeds this into a Bidirectional transformer. The key innovation is adapting the image features as additional tokens to the transformer model. Supervised Multimodal Bitransformers for Classifying Images and Text Additionally, there are models, (Lu et al. 2019)  and (Su et al. 2020), which define pre-training tasks for images and text. Both models pre-train on the , which contains roughly 3.3 million image-caption pairs (web images with captions from alt text). In both cases, for a given image, a pre-trained object detection model like Faster R-CNN obtains vector representations for regions of the image, which count as input token embeddings to the transformer model. ViLBERT VLBert Conceptual Captions dataset As an example, ViLBert pre-trains on the following training objectives: : Mask input image and word tokens. For the image, the model tries to predict a vector capturing image features for the corresponding image region, while for text, it predicts the masked text based on the textual and visual clues. Masked multimodal modelling : Whether the image and text pair are actually from the same image and caption pair. Multimodal alignment An image of the pretraining tasks and an example of masked multimodal learning are shown below. Given the image and text, if we mask out dog, the model should be able to use the unmasked visual information to correctly predict the masked word to be dog. All these models use the bidirectional transformer model that is the backbone of BERT. The differences lie in the pretraining tasks the models are trained on and slight additions to the transformer. In the case of ViLBERT, the authors also introduce a co-attention transformer layer (shown below) to define the attention mechanism between the modalities explicitly. Finally, there’s also (Tan and Mohit 2019), another pre-trained transformer model that as of Transformers version 3.1.0, is implemented as part of the library. The input to LXMERT is the same as ViLBERT and VLBERT. However, LXMERT pre-trains on aggregated datasets, which also include visual question answering datasets. In total LXMERT pre-trains on 9.18 million image text pairs. LXMERT Transformers on Aligning Audio, Visual, and Text Beyond transformers for combining image and text, there are multimodal models for audio, video, and text modalities in which there is a natural ground truth temporal alignment. Papers along this direction include MulT, (Tsai et al. 2019), and the Multimodal Adaptation Gate (MAG) from (Rahman et al. 2020). Multimodal Transformer for Unaligned Multimodal Language Sequences Integrating Multimodal Information in Large Pretrained Transformers MuIT is similar to ViLBert in which co-attention is used between pairs of modalities. Meanwhile, MAG hopes to inject other modality information at certain transformer layers via a gating mechanism. Transformers with Text and Knowledge Graph Embeddings Some works have also identified knowledge graphs as a vital piece of information in addition to text data. (Ostendorff et al. 2019) uses features from the author entities in the Wikidata knowledge graph in addition to metadata features for book category classification. In this case, the model is a simple concatenation of these features and BERT output text features of the book title and description before some final classification layers. Enriching BERT with Knowledge Graph Embeddings for Document Classification Key Takeaway The main takeaway for adapting transformers for multimodal data is to ensure that there's an attention or weighting mechanism between the multiple modalities. The attention mechanisms can occur at different points of the transformer architecture, as encoded input embeddings, injected in the middle, or combined after the transformer encodes the text data. Multimodal Transformers Toolkit Using what we’ve learned from the literature review and the comprehensive HuggingFace library of state of the art transformers, we’ve developed a toolkit. The package extends any HuggingFace transformer for tabular data. To see the code, documentation, and working examples, check out . multimodal-transformers the project repo At a high level, the outputs of a transformer model on text data and tabular features containing categorical and numerical data are combined in a combining module. Since there is no alignment in our data, we choose to combine the text features after the transformer’s output. The combining module implements several methods for integrating the modalities, including, attention and gating methods inspired by the literature survey. More details of these methods are available . here Walkthrough Let's work through an example where we classify clothing review recommendations. We'll use a simplified version of the example included in the Colab notebook. We will use the from Kaggle, which contains 23,000 customer reviews. Women's E-Commerce Clothing Reviews In this dataset, we have text data in the and columns. We also have categorical features from the , , and columns and numerical features from the and . Title Review Text Clothing ID Division Name Department Name Class Name Rating Positive Feedback Count Loading The Dataset We first load our data into a , which works with PyTorch's data loaders that include the text inputs for HuggingFace Transformers and our specified categorical feature columns and numerical feature columns. For this, we also need to load our HuggingFace tokenizer. TorchTabularTextDataset Loading Transformer with Tabular Model Now we load our transformer with a tabular model. We first specify our tabular configurations in a object. This config is then set as the member variable of a HuggingFace transformer config object. Here, we also specify how we want to combine the tabular features with the text features. In this example, we will use a weighted sum method. TabularConfig tabular_config Once we have the set, we can load the model using the same API as HuggingFace. See the for the list of currently supported transformer models that include the tabular combination module. tabular_config documentation Training For training, we can use HuggingFace’s class. We also need to specify the training arguments, and in this case, we will use the default. trainer Let's take a look at our models in training! Results Using this toolkit, we also ran our experiments on the dataset for recommendation prediction and the dataset for price prediction. The former is a classification task, while the latter is a regression task. Our results are in the table below. The text_only combine method is a baseline that uses only the transformer and is essentially the same as a HuggingFace for SequenceClassification model. Women’s E-Commerce Clothing Reviews Melbourne Airbnb Open Data We can see that incorporating tabular features improves performance over the text_only method. The performance gains also depend on how strong the training signals from the tabular data are. For example, in the review recommendation case, the text_only model is already a strong baseline. Next Steps We've already used the toolkit successfully in our projects. Feel free to try it out on your next machine learning project! Check out the and the included script for how to do evaluation and inference. Also, if you want support for your favourite transformer, feel free to add transformer support . documentation main here Appendix Readers should check out and , for a well summarized overview of transformers and BERT. The Illustrated Transformer The Illustrated BERT Here’s a quick taxonomy of papers we reviewed. Transformer on Image and Text Supervised Multimodal Bitransformers for Classifying Images and Text  (Kiela et al. 2019) ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks (Lu et al. 2019) VL-BERT: Pre-training of Generic Visual-Linguistic Representations (Su et al. ICLR 2020) LXMERT: Learning Cross-Modality Encoder Representations from Transformers (Tan et al. EMNLP 2019) Transformers on Aligning Audio, Visual, and Text Multimodal Transformer for Unaligned Multimodal Language Sequences (Tsai et al. ACL 2019) Integrating Multimodal Information in Large Pretrained Transformers (Rahman et al. ACL 2020) Transformers with Knowledge Graph Embeddings Enriching BERT with Knowledge Graph Embeddings for Document Classification (Ostendorff et al. 2019) ERNIE: Enhanced Language Representation with Informative Entities (Zhang et al. 2019 [ Colab Tutorial ] [ Github ] Also published at https://medium.com/georgian-impact-blog/how-to-incorporate-tabular-data-with-huggingface-transformers-b70ac45fcfb4

HUGGINGFACE

huggingface.co

50-200 employees

Since 2016

COMPANY RANKING

HUGGINGFACE

EVERGREEN INDEX #6335

Representing this company? Click here to claim and customize this page!

HuggingFace WIKI

HUGGINGFACE

huggingface.co

50-200 employees

Since 2016

COMPANY RANKING

HUGGINGFACE

EVERGREEN INDEX #6335

HACKERNOON STORIES ON HUGGINGFACE

HUGGINGFACE NEWS & MENTIONS

Representing this company? Click here to claim and customize this page!

HuggingFace WIKI