GenOs Index - The Q3 Edition: The Trends that Hold Strong and the Projects that Stand the “Test of Time”

We are back with the latest installment of the Generative AI Open Source (GenOS) Index showcasing the fastest growing open source projects in Generative AI in Q3 (July to September)!

First, a quick refresher on the methodology. Every quarter, we identify the top 30 open source projects in Generative AI as ranked by GitHub star growth (adds) in the preceding 90 days, with 500-star adds being the minimum for a project to be considered. Furthermore, we categorize the projects into Applications, Infrastructure/Tools and Models to provide visibility into how different parts of the Generative AI ecosystem are evolving.

The key takeaways from the Q3 GenOS Index are as below.

The top 30 projects continue to see significant turnover

In keeping with the fast pace of the GenAI space, the current GenOS Index has 18 new entrants, up from 13 new entrants in the previous edition of the Index. Perhaps more remarkably, 6 of the top 10 projects are new entrants with the top 2 - Open Intepreter (OpenAI's Code Interpreter running locally) and SHAP (a game theoretic approach to explain the output of any machine learning model) - appearing on the GenOS Index for the first time ever.
‍

Infrastructure/Tools surges ahead of the other categories

In the previous editions of the GenOS Index, the top 30 projects were fairly equally distributed across the Applications, Infrastructure/Tooling and Models categories. In the Q3 edition, however, we see Infrastructure/Tooling dominating by being over half (53%) of the top 30. With AI projects increasingly getting deployed in production, we believe that projects in the Infrastructure/Tools category are getting more attention from the open source community than ever before.

Three trends that hold strong

Looking back at all the editions of GenOS Index since its launch, we see the following three trends continuing to hold strong, if not gaining momentum:

• Running models locally. Two new entrants in the current Index reinforce that trend: Open Interpreter (#1) that lets LLMs run code (Python, Javascript, Shell, and more) locally, and Ollama (#20) that helps run Llama2 on a laptop, applying a linux container like construct to run LLMs.
• Privacy and data security matters. LlamaGPT (new entrant at #15), LocalGPT (#21) and PrivateGPT (#29) all essentially promise the same: the ability to interact privately with your documents using the power of GPT with zero data leak.
• Agents continue to proliferate. Besides AutoGPT (#18) continuing its run in the top 30, we have two new entrants - Generative Agents (#6) and GPT Engineer (#10) - in the agents category.

“Test of Time” projects

While we have had multiple projects move in and out of the GenOS Index over time, we have now started tracking the projects that, once they got on the Index, have stayed on. We call those the “Test of Time” projects and that select group includes: 

• LLaMA - Inference code for LLaMa models
• Stable Diffusion Web UI - A browser interface based on Gradio library for Stable Diffusion
• LangChain - Building applications with LLMs through composability
• LlaMA.cpp - Inference of LLaMA model in C/C++
• Text Generation Web UI - A Gradio web UI for Large Language Models
• AutoGPT - Go-to toolkit for supercharging agents
• Transformers - State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX
• PrivateGPT - Interact privately with documents using the power of GPT

The list of all top 30 projects in this month’s GenOS Index are as follows:

The Rising Stars

As we had done in the past, we highlight below 5 other really interesting projects that, while not on the GenOS Index this month, have gained significant traction and are anticipated to break into a future edition of the GenOS Index:

• Gorilla from UC Berkeley enables LLMs to use tools by invoking APIs. Given a natural language query, Gorilla comes up with the semantically and syntactically correct API to invoke, and currently supports thousands of (and growing) API calls while reducing hallucination.
• SuperAGI is a developer-first autonomous AI agent framework enabling developers to build, manage and run autonomous agents quickly and reliably.
• FaceChain is a Deep Learning toolchain powered by ModelScope that generates a digital twin (personal portraits) in different settings from even a single portrait photo. 
• DevOpsGPT combines LLMs with DevOps tools to convert natural language requirements into working software, aiming to improve development efficiency, shorten development cycles, and result in higher-quality software delivery.
• DSPy from Stanford NLP is the framework for solving advanced tasks with language models (LMs) and retrieval models (RMs). It unifies techniques for prompting and fine-tuning LMs, and approaches for reasoning, self-improvement, and augmentation with retrieval and tools.

That is it for this edition of the GenOS Index. Stay tuned for the Q4-2023 GenOS Index!