Strategies & Best Practices Every Firefly Services API Developer Should Know | by Kelvin Xu | Aug, 2024
Adobe Firefly Services offers a comprehensive set of generative AI and creative APIs that can enhance enterprise processes and workflows. The services include APIs such as Firefly APIs, Photoshop APIs, Lightroom APIs, and Content Tagging APIs. These APIs, having evolved from different backgrounds and timelines, exhibit some differences in API signatures, input/output formats, and response behaviors. It’s crucial to understand these nuances and adopt key strategies and best practices that can guide developers in using these APIs effectively.
Authentication is the foremost step before making any valid API request. Licensed customers can follow the guide here to obtain the client ID and client secret, then use a curl command to acquire the access token. This process is fairly straightforward, but it’s important to note that this is a server-to-server authentication mechanism, meaning that it does not involve your application’s end users logging in. The authentication occurs solely between your application and Firefly Services, so it’s essential to implement it securely.
For example, in web application development, developers must avoid obtaining the access token from the client side, as this could easily expose credentials. The access token should be fetched on the server side to ensure that credentials remain secure.
When implementing this in practice, consider using a common library or method to generate the token centrally. If you’re using Node.js, you can leverage this library. Since the token is valid for 24 hours, it’s wise to cache it (e.g. with a TTL of 23 hours and 55 minutes) and reuse it rather than fetching a new token every time.
Both Photoshop (including Lightroom) and Firefly APIs generally require input and output images to be served from cloud storage or pre-signed URLs. These APIs currently support S3, Azure Blob Storage, and Dropbox. For Photoshop APIs, a resource served from a pre-signed URL is required to start any API operation. Firefly, however, provides an upload endpoint for temporarily storing source files if no cloud storage is available. This upload endpoint returns an upload ID rather than a pre-signed URL, which means it cannot be reused in Photoshop (and Lightroom) APIs. Therefore, if your implementation involves orchestration across Firefly and Photoshop APIs, it’s advisable to have cloud storage ready from the start.
As of now, Firefly APIs are synchronous, meaning that after making a request, you will receive the results (either success with output images or failure with an error message) in the response within a few seconds. This real-time generation does require some computational time.
On the other hand, Photoshop APIs are asynchronous. After making a request, you will immediately receive a job ID rather than the actual outputs. You will then need to poll the job status using the job ID until you receive the results.
Understanding these API characteristics will help you design and develop your applications more effectively (Ideally, both APIs should support both synchronous and asynchronous operations, or at least offer a choice).
For real-time generation apps using Firefly APIs, you may need to set up appropriate expectations upfront, perhaps by using a loading spinner or skeleton loader on the UI side. For Photoshop APIs, asynchronous processing is excellent for parallel processing at scale. However, for real-time user interactions, you might need to “convert” the asynchronous behavior to synchronous by introducing status polling until completion, or by using the Firefly SDK, which encapsulates the polling process for you.
For scalable parallel processing, you also can leverage eventing. Photoshop supports eventing through Adobe’s serverless platform, Adobe IO Events.
In this model, the client doesn’t need to poll the status continuously, as a job completion event will be emitted to IO Events, to which a webhook or IO runtime action can subscribe. This approach allows the client to be more performant and is ideal for implementing bulk processing (the webhook or IO runtime action running in Adobe IO platform is autoscaled).
By default, Firefly APIs have a rate limit of 4 RPM (requests per minute) and 9,000 RPD (requests per day). Licensed customers can request higher limits based on their annual quota and estimated traffic. However, despite the best estimations, hitting the rate limit might be inevitable during unexpected traffic spikes, or a large campaign event.
A common strategy is to implement a retry logic. If you encounter a 429 response code, you can retry after waiting for the “retry-after” period indicated in the response header or implement an exponential backoff pattern.
Another strategy to avoid hitting the rate limit is pre-generation (in the case of text2image). The enterprises wanting to enforce brand consistency in generated images should also consider this strategy. The process can be split into pre-generation and delivery phases.
During the pre-generation phase, enterprises can use predefined prompt templates to generate large number of images that maintain brand consistency (techniques including style/structure referencing or customer models, etc.). This can be done over an extended period, respecting the rate limit, and storing the generated images in a database or cloud storage with search and query capabilities. Metadata and tagging can also be generated (i.e. using Content Tagging API or GPT Vision) to assist in indexing.
On the delivery side, client apps can guide users to turn their inputs into image queries against the stored images. These pre-generated images can be delivered directly or used to generate more personalized experiences based on user inputs. Since the images are pre-generated, quality and brand consistency can be ensured, real-time inference is avoided (cost effective), and hitting the rate limit is mitigated.
