Using Arcus
Now that we’ve established the key concepts of the Arcus Data Exchange, let’s take a look at how you can use Arcus to improve your ML performance.
Projects are the primary way to consume external data from the Arcus Data Exchange. A project represents an individual AI application and corresponds to a single instance of using external data from the exchange.
Arcus supports a variety of project types which correspond to the different product offerings:
- Model Enrichment: A model enrichment project integrates directly with your predictive or classification models to enrich the model with additional signals, features and samples. Arcus understands your application and first-party data to match you to relevant external data and trials the different data candidates provided by the exchange to empirically understand how they affect your model performance. Once you select a data candidate, it is seamlessly used across your training and inference workflows.
- Prompt Enrichment: A prompt enrichment project integrates directly with your Generative AI application to perform data augmented generation by enriching your prompt with relevant external data signals. This context improves the quality of your generated results by grounding it in real-world context.
To get started using Arcus, you’ll need to create a project (get early platform access here). When you create a project, you’ll be asked to provide some basic information about your application and the requirements for the data you want to consume. This information is used to connect you to the most relevant data in the Arcus Data Exchange.
Once you’ve created a project, you’ll be able to see the status of your project in the Arcus UI. The status of your project will change as the project progresses through the different stages of the Arcus workflow.
The next stage is to connect your AI application to Arcus. You can do this by integrating Arcus into your ML workflow using the Arcus client libraries. This library transparently wraps popular ML frameworks, while enriching the underlying model or prompt with external data from the Arcus Data Exchange. It takes less than 10 lines of code to integrate Arcus into your existing ML workflows.
Once you’ve connected to Arcus, the Arcus Data Exchange matches your application to the most relevant external data candidates on the exchange and compares the performance of your application using the different candidates through the trialing process. Trialing will empirically train or run your model with the enriched data to determine how your model performance is affected.
Based on these results, you can select the best candidate to use in your application and seamlessly integrate the data into your application’s ML workflow for downstream usage (e.g. for model training and inference).
Once you’re ready to deploy your enriched model or prompt, Arcus only charges you for the data you use. The cost of a data candidate is expressed as credits per 1000 data accesses (used during your training and inference workflows), determined during the auction process.
Now that you’ve learned the basics of the Arcus Data Exchange, you’re ready to get started using Arcus. To get started, get early access to the platform and create a project and integrate your application with Arcus using the Arcus Python SDK. Read on to learn more about the different ways you can use Arcus to improve your AI applications: