Keap / Infusionsoft API Key Sunsetting: What you need to know

#Summary

Since 2014, WP Fusion has used an “API key” to connect your WordPress site to Infusionsoft and Keap. This was the key you generated and entered into the Setup tab when first installing WP Fusion.

On June 12th, 2024, Keap announced they were removing support for API Keys (now called “Legacy API Keys”) and all apps, including every website running WP Fusion, will need to update to a new authentication method.

Sites using legacy API keys would begin “scheduled service disruptions” on August 5th.

#Why did Keap do this?

We don’t really know. They have made some valid points about needing better insights into the way their APIs are used, and managing API permissions within accounts, but this doesn’t explain the accelerated timeline.

For example when HubSpot makes a potentially breaking API change, we are alerted a year in advance, which gives us plenty of time to make changes and test everything for compatibility.

Keap gave us two months to not only update our integration, but also get it out to the 2,200+ customers who have used WP Fusion to connect to Keap over the last year.

I don’t see any good reason for forcing this change through with so little notice. It ultimately hurts Keap’s customers, who are now either scrambling to hire developers to update their old integrations (going back 18 years), or they’re still unaware of the changes and won’t realize it until their integrations stop working 😞

#When will the changes go into effect?

Keap is scheduling a series of “brownouts” to help customers identify which integrations are still using legacy API keys. Essentially, they turn off the API for a few hours at a time, in the hopes that you notice something is broken.

The brownout schedule is available here:

Legacy API keys will be completely disabled on Thursday, Oct. 31st at 7am Phoenix time (UTC 14:00, London 2pm, Sydney Fri 1am). 

#What changes have we made at WP Fusion?

On June 18th we began the process of re-writing WP Fusion’s Keap integration to support the new Service Account Key authentication method, and also moving from the older XMLRPC API to the newer REST API.

Since Keap has hinted that the XMRPC API may be removed soon as well, we decided to make both changes at once to avoid ongoing disruptions.

We had three main priorities:

• Maintain support for legacy API keys: Many folks have the WP Fusion plugin set to automatically update. We couldn’t introduce an update that disconnected Keap until you generated a new key. We needed the integration to continue working until you had time to read the notice in the admin, our email alert, or this blog post, and make the necessary changes.
• Maintain support for legacy settings: The new API uses an entirely different structure for referencing custom fields, especially addresses and phone numbers. We needed to make sure all your legacy API field mappings across the WP Fusion settings, your form feeds, and products would continue to work without requiring a manual update.
• Preserve backwards compatibility: Many developers have built custom integrations on top of WP Fusion that reference the old API via wp_fusion()->crm->app. We needed to maintain backwards compatibility on these sites by adapting all of the old iSDK methods to use the new REST API and authentication.

This process took a little over a month before we began testing, and involved 1,834 lines of new code.

It was basically a complete re-write from the ground up of an integration we’d been tweaking and optimizing for the last ten years 😩

#Why we went with Service Account Keys

Infusionsoft offers two methods of authentication: OAuth, and Service Account Keys.

OAuth may be familiar if you’ve integrated other services with Keap. You click an “Authorize” button and are redirected to Keap to approve the app and grant permissions. The app then shows in your account as connected, with the name of the provider (like, “Zapier”).

Service Account Keys are the closest equivalent to the legacy API key. Each key is unique to your account. You generate it, and then copy-paste it into the WP Fusion settings to use it. The key can only be seen by you, and all data is communicated directly between your site and Keap.

While OAuth provides a better setup flow, and easier management for the app developer, it has some downsides on a WordPress site:

1. OAuth apps have a limit of 1,500 API calls per minute, or 150,000 calls per day. This doesn’t sound like a lot until you consider that those limits are shared between everyone using the WP Fusion app.

A SaaS app provider like Zapier can see the total number of API calls they’re sending to Keap and request an API limit increase before it causes a problem.

With WP Fusion, your data is sent directly from your site to Keap. We have no visibility into the remaining API calls in the quota, since this data is only sent back to the site that initiated the API call.

This means that if a new WP Fusion customer signed up and exported 300,000 WooCommerce orders (at roughly 5 API calls per order), the quota would be hit, and WP Fusion would be disconnected on every customer website until we were able to contact Keap support and request an increase 😩.

This wasn’t an acceptable risk for us, so we decided not to implement OAuth at the current time (we may add an option in the future).

2. OAuth Access Tokens expire every 24 hours. After 24h, they need to be “refreshed” to get a new token. While this is largely an automated process, and we do support it with many CRMs, it introduces extra complexity.

We can’t reliably schedule an event to happen on your website exactly every 24h, since different hosts use different event scheduling techniques.

What WP Fusion does is waits until there is an API error indicating that the token has expired, and then requests a new access token at that time.

This mostly works well but can be messed up by caching, or by copying your site to staging without activating “staging mode“. In that case the token is refreshed to the staging site and can no longer be refreshed on the live site.

Once that happens, the only solution is to log into your WordPress admin and manually re-authorize WP Fusion with Keap. Any data synced while the token was expired will be lost and will need to be exported manually using one of our export tools.

Memberium has developed a clever solution to this problem by using OAuth but setting a Service Account Key as a fallback in case OAuth gets disconnected, but we were already crunched for time due to the looming brownouts and were apprehensive about building a new authorization system in parallel.

#What do you need to do to update?

WP Fusion 3.44.0 was released via automatic update on August 5th, with support for the new API and authentication.

After updating, you will see a banner across the top of your admin prompting you to update to a new Service Account Key.

Log into your Infusionsoft / Keap account and head to Settings » API Settings » Service Account Keys, and click New Key to add a new key.

We recommend generating a separate key for each of your sites to avoid API throttling (more on that below).

Paste the key into the Setup tab of the WP Fusion settings, and save the settings. That’s it! We tried to make it as easy as possible 😅

#Fixed issues

I won’t sugarcoat it, this was a rough rollout 😞.

Normally we’d like to do a limited beta release to some Keap power users, to test on staging sites, before we did a general release— but we only had about five days to test the changes in-house before the first scheduled service interruption.

I want to personally apologize to anyone who updated last Monday and ran into the issues below. And also thank you, for being so proactive in reporting the bugs, and giving us access to the logs so we could jump right into fixing them 🙏

We’ve released four updates since v3.44.0 to deal with unexpected problems coming from the new API. Those are:

#3.44.0.1 – 8/6/2024

• Fixed new integration not loading more than 1000 each of tags or tag categories
• Fixed new integration not importing all contacts if no tag was specified for the import

#3.44.0.2 – 8/7/2024

• Fixed custom fields with spaces in the labels not migrating to the new API field mapping
• Fixed “Unprocessable entity” errors when syncing legacy custom fields with spaces in the label

#3.44.1 – 8/12/2024

• Added ISO 3166-1 country name conversion for “United States” to “USA”. It previously only matched “United States of America”, and was generating errors if you synced “United States” to a country field as a value.
• Fixed the billing and shipping addresses getting swapped in some instances when updating contacts
• Fixed date-time fields not being synced in the ISO8601 format
• Fixed new API not loading more than 10 available products (for Enhanced Ecommerce)
• Fixed new orders created with Enhanced Ecommerce adding a pseudo-item “wpf_rest_product” to the Products list in Keap

#3.44.1.1 – 8/15/2024

• Added US state name to ISO 3166-2 code conversion for updating billing and shipping addresses
• Fixed date fields (like Birthday) not syncing in the Y-m-d format
• Fixed date-time fields loaded from Keap not being re-formatted into the WordPress date-time format

#Outstanding issues

#Contact Notes

Previously, Keap allowed you to update the Contact Notes field over the API, and this would append the note data to any existing notes in that field.

The new API replaces all of the notes in that field whenever a note is synced. For the moment we’ve disabled syncing this field for existing contacts to avoid data loss (it can still be set for new contacts).

We are currently working on some solutions to allow you to continue appending notes to that field.

#International billing and shipping addresses

Keap is very strict about the formats used over the API for countries and regions (for example New York must be synced as US-NY).

We have developed a conversion system for every country name and abbreviation over to the new standard, as well as name and abbreviation conversion for US states.

We aren’t sure yet which other countries require ISO 3166-2 compatible abbreviations for region, state, or province names. It’s possible that non-US shipping or billing addresses synced to Keap may display with a missing State field.

Please contact us if you notice this happening in your account.

#Ecommerce order line items

WP Fusion’s Enhanced Ecommerce addon is tested with the new API for creating products and orders, but the new API no longer supports line items like shipping, taxes, discounts, and fees.

If you attempt to create an order with line items, it will not be synced, and you’ll see an error “Product id 0 is invalid”.

We are working on a fix for this for Monday’s update of the core WP Fusion plugin.

#Syncing user names and passwords

Infusionsoft has two fields on the Additional Info tab in the contact record for User Name and Password.

These fields were part of the now-discontinued CustomerHub integration (it was shut down in 2019) and weren’t ever intended for use over the API, especially since the password field will throw an error if the password is longer than 16 characters 🤦.

We’ve noticed many WP Fusion users using these fields to store usernames and passwords from their WordPress members.

Infusionsoft has completely removed these fields from the new REST API, and they can no longer be updated.

If you attempt to update the User Name or Password field, WP Fusion will record a notice to the logs, and the data will not be synced (to avoid an API error).

If you need to sync passwords with Infusionsoft / Keap (which we don’t recommend for security reasons), you will need to create new text custom fields to store this data, and map them again in the WP Fusion settings.

It is not possible (again, for security reasons), to export the passwords of your existing members. Passwords will only sync for new user registrations and during password resets.

#API Quotas Exceeded

The new Service Account Key allows for the following limits:

• 10 Queries per Second
• 240 Queries per Minute
• 30000 Queries per Day (Resets at 12:00 AM UTC)

Please note that these are significantly lower than the 160,000 queries per day allowed by the legacy API key.

We haven’t yet run into anyone hitting the actual limits, but we have some ideas on how to handle it in case it comes up: for example adding a backup SAK, or authenticating via OAuth.

However, during testing we have run into two cases where an API call failed with a Quota Exceeded message, even when the quota was not exceeded.

Below is the API response we received from Keap during one of these tests.

It clearly shows that only one API call had been used, and the full quota remained, so we’re not sure why it’s throwing an error.

(
    [headers] => WpOrg\Requests\Utility\CaseInsensitiveDictionary Object
        (
            [data:protected] => Array
                (
                    [content-type] => application/json
                    [x-keap-product-spike-limit] => 10ps
                    [x-keap-product-throttle-limit] => 240
                    [x-keap-product-throttle-time-unit] => minute
                    [x-keap-product-throttle-interval] => 1
                    [x-keap-product-throttle-available] => 239
                    [x-keap-product-throttle-used] => 1
                    [x-keap-product-quota-limit] => 150000
                    [x-keap-product-quota-time-unit] => day
                    [x-keap-product-quota-interval] => 1
                    [x-keap-product-quota-available] => 149862
                    [x-keap-product-quota-used] => 138
                    [x-keap-product-quota-expiry-time] => 1723680000000
                    [x-keap-tenant-id] => ps359.infusionsoft.com
                    [x-keap-tenant-throttle-limit] => 5000
                    [x-keap-tenant-throttle-time-unit] => minute
                    [x-keap-tenant-throttle-interval] => 1
                    [x-keap-tenant-throttle-available] => 4999
                    [x-keap-tenant-throttle-used] => 1
                    [access-control-allow-credentials] => true
                    [access-control-allow-origin] =>
                    [x-request-id] => cc0928de-3340-4c44-aaf6-ee9b6541baa9
                    [content-length] => 212
                    [date] => Wed, 14 Aug 2024 13:38:22 GMT
                    [via] => 1.1 google
                    [alt-svc] => h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
                )
 
        )
 
    [body] =>
                {
                  "code": "429",
                  "message": "Quota Exceeded",
                  "status": "Request Throttled",
                  "details": null
                }
 
    [response] => Array
        (
            [code] => 429
            [message] => Too Many Requests
        )
 
)

In both cases we were testing updating the same contact multiple times in quick succession via a Gravity Forms submission, so it’s possible Keap flagged this as inappropriate and blocked the API call.

If that’s the case it shouldn’t come up often on live sites, but please let us know if you run into any Quota Exceeded messages with WP Fusion, and we will pass them on to Keap API support for further analysis.

#Downgrading WP Fusion

If you’re experiencing issues with the new integration, you can downgrade WP Fusion to v3.43.20.1 from the Previous Versions section in your account on our site.

v3.43.20.1 was the last update before the API changes and will restore the previous functionality.

Please make sure to update the settings to use your old Legacy API Key, as the new Service Account Key won’t work with the old API.

Also note that by downgrading you will be affected by the scheduled brownouts. During that time, no data will sync with Keap, and errors will be recorded to the WP Fusion logs.

There is currently no way to re-process failed API calls when the brownout ends. You will need to manually run export operations for any missing data, for example LearnDash course progress or WooCommerce orders.

If you do have to downgrade, please contact us and let us know the issues you were facing so we can fix them in the next update 🙏