Hugging Face has released Idefics2, an open-source multimodal vision-language model with 8B parameters under the Apache-2.0 license. It supports arbitrary interleaved image-text inputs and text generation, featuring enhanced OCR capabilities and native-resolution image processing. It outperforms models of similar size on benchmarks such as MMMU and is integrated into 🤗 Transformers. Accompanying resources include the open-source instruction-tuning dataset The Cauldron and a complete training data manifest.
Hugging Face has launched a new "Deploy on Google Cloud" integration, supporting one-click deployment of open-source foundation models tagged with "text-generation-inference" from Hugging Face Hub (such as Zephyr Gemma) onto Vertex AI or GKE.
Google has launched the CodeGemma series of open-source code large language models, including a 2B base model (focused on code completion), a 7B base model (trained on mixed code and natural language), and a 7B instruction-tuned model (supporting code dialogue). All are based on Gemma pre-trained checkpoints and additionally trained on 500B tokens, with an 8K context window.
Hugging Face announced a partnership with cloud security company Wiz to integrate its Vulnerability Management and CSPM capabilities to enhance platform security. The Wiz research team discovered arbitrary code execution risks related to pickle in Hugging Face's sandboxed computing environment; all related issues have been fully remediated. Hugging Face is also promoting Safetensors as a secure alternative to pickle and continues to drive security best practices within the open-source AI community.
In a blog post, Hugging Face introduced how to use 🤗 Optimum Intel to perform post-training static quantization (INT8) on SetFit models, achieving up to a 7.8x improvement in inference throughput on Intel Xeon CPUs.
Hugging Face announced the launch of the Deploy on Cloudflare Workers AI integration, supporting serverless invocation of Llama 2 7B at Cloudflare edge data centers.
The Pollen Robotics team has released the open-source library pollen-vision, which provides a unified interface for zero-shot vision models such as OWL-VIT, Mobile SAM, and RAM. It supports 3D object localization (x, y, z coordinates) based on RGB images and text inputs, suitable for the Reachy humanoid robot's task of grasping unknown objects. The library can run in real time on consumer-grade GPUs (such as RTX 3070) and has been open-sourced on GitHub.
The Hugging Face blog introduces binarization (binary) and int8 scalar quantization methods for embedding vectors. Demonstrated in real-world retrieval scenarios on 41 million Wikipedia texts, quantization can achieve up to a 45x retrieval speedup.
Lighthouz AI and Hugging Face jointly launched Chatbot Guardrails Arena, a stress-testing platform for AI chatbot privacy protection capabilities; users can interact with two anonymous...
GaLore is a memory optimization technique for large language model training that significantly reduces optimizer state memory usage via low-rank gradient projection (reducing optimizer state memory during training by over 82.5%), enabling the training of Llama architecture 7B parameter models on consumer-grade GPUs such as the NVIDIA RTX 4090.
Hugging Face released Cosmopedia, an open-source synthetic dataset for LLM pretraining containing over 30 million files and 25 billion tokens, generated by Mixtral-8x7B-Instruct-v0.1.
Hugging Face officially announced the launch of the Train on DGX Cloud service, available to Enterprise Hub organizations, supporting the fine-tuning of Llama using H100 or L40S GPUs on NVIDIA DGX Cloud.
Hugging Face has released Quanto, a PyTorch quantization backend that supports int2/int4/int8/float8 weights and int8/float8 activations. It runs on CPU/GPU/MPS devices, is compatible with torch.compile, and has been integrated into the transformers library, accessible directly via QuantoConfig.
Hugging Face released the WebSight dataset. Version 0.1 contains 823,000 pairs of synthetic screenshot/HTML data. Version 0.2 was upgraded to 2 million pairs, using real screenshots and switching to Tailwind CSS. The Sightseer model, fine-tuned on this dataset, can convert webpage screenshots into HTML code with matching images. Resources include the dataset, a technical report (arXiv:2403.09029), and Colab examples.
Hugging Face and Intel jointly released CPU-optimized embedding models based on Optimum Intel and IPEX (such as int8 static quantized versions of BGE-small/base/large).
Hugging Face, in collaboration with researchers from the University of California, Los Angeles (UCLA), released the ConTextual dataset, which contains 506 instructions requiring joint reasoning based on image and text cues in a context-sensitive manner, covering time reading.
Hugging Face and Argilla have launched a collaborative dataset building workflow based on the integration of Spaces and Argilla, allowing users to participate in annotation within seconds using their Hugging Face accounts. This workflow has been used to generate the 10k_prompts_ranked dataset, which contains 10,000 prompts with user ratings. The first community co-building program is now open, offering free storage, enhanced CPU Spaces, and joint promotion support.
Hugging Face published a blog post announcing support for Intel® Gaudi® 2 AI accelerators in its text generation pipeline, compatible with Llama-2-7b, 13b, and 70b models.
BigCode has released the StarCoder2 series of open-source code large language models, available in three sizes: 3B, 7B, and 15B, all trained on the newly released The Stack v2 dataset.
Hugging Face has launched TTS Arena, a text-to-speech model evaluation platform inspired by LMSys Chatbot Arena. It allows users to submit text, listen to speech generated by two models, and vote for the more natural one; results are compiled into a public leaderboard using an Elo-like algorithm. Initial models include ElevenLabs, MetaVoice, OpenVoice, Pheme, WhisperSpeech, and XTTS.
Mistral AI has released its conversational assistant, le Chat, which supports Mistral Large, Mistral Small, and the prototype model Mistral Next. It offers multilingual interaction and system-level adjustable moderation mechanisms, while simultaneously launching a self-deployed version for enterprises, le Chat Enterprise.