Exporting Data / Batch Operations

Overview

WP Fusion includes a series of tools that let you export data in bulk to your CRM. These are found under Settings » WP Fusion » Advanced » Batch Operations.

The options available will depend on which plugins you have active. For each operation you can hover over the tooltip indicator to get a description of what data will be synced or modified.

Note: These operations can potentially modify thousands of records in both WordPress and your CRM, and they are irreversible. It’s always recommended to test with a single record before running a batch operation— for example doing a test checkout before exporting orders, or a test registration before exporting users.

Usage

To run a batch operation, select the operation you’d like to perform, and then click Create Background Task. A status indicator will appear at the top of the page showing the number of records to be processed and the number remaining.

Some notes:

  • You can leave the page while the background worker runs and it will continue to process the queue in the background.
  • If the background process gets stuck you can refresh the page and it will pick up where it left off.
  • It’s recommended to turn on the activity logs so you can see what data is being sent or loaded, as well as any potential errors.

Note on speed: Most CRMs have some kind of API limits with regards to the number of API calls you can make in a period of time. WP Fusion pauses in between each step in the background process to avoid exceeding these API limits. How long the pause is depends on your connected CRM.

Depending on the number of records you are processing, this can cause the background worker to take several hours (or even days) to complete. For example exporting 30,000 WooCommerce orders to Drip would take about 18 hours to fully process.

Core methods

These operations are part of WP Fusion core and aren’t tied to any particular plugin integration. They are:

Resync Contact IDs and Tags

All WordPress users will have their contact IDs checked / updated based on email address and tags will be loaded from their CRM contact record.

When to use it: When a user registers in WordPress, WP Fusion stores their CRM contact ID on your site so that future updates can properly be synced. But if you manually merge a bunch of contacts (for example to remove duplicates), it’s possible that these IDs will no longer be accurate. Running a Resync Contact IDs and Tags operation ensures WP Fusion has all of your WordPress users linked to the correct contact record.

Resync Tags

Loads updated tags from your CRM for all WordPress users who already have a saved contact ID. This also triggers any automated enrollments via linked tags.

When to use it: If you’ve modified a lot of tags on a lot of contacts in your CRM (or deleted tags), this operation will bring the tags cached in WordPress for your users up to date with what’s in your CRM. This operation is faster than Resync Contact IDs and Tags because it doesn’t need to first confirm the contact’s email address.

Also, if you’ve recently changed an auto-enrollment tag on a course or membership (for example with a LearnDash course), you may want to update your users automated enrollments based on their current CRM tags. Running the Resync Tags operation will trigger any automated enrollments (and un-enrollments) when the tags are loaded from your CRM.

Export Users

All WordPress users without a matching CRM contact record will be exported as new contacts. Any fields configured on the Contact Fields tab will be synced.

When to use it: If you’re just setting up WP Fusion for the first time, you may have WordPress users that aren’t already in your CRM or marketing automation tool. This tool lets you export those users.

Push User Meta

All WordPress users with a contact record will have their meta data pushed to your CRM, overriding any data on the contact record with the values from WordPress. Any fields enabled on the Contact Fields tab will be synced.

When to use it: This is most commonly used when you’ve just enabled a new field for sync (for example User ID, or Date Registered), and need to export the values from that field to existing contact records in your CRM.

Pull User Meta

All WordPress users with a contact record will have their meta data loaded from your CRM, overriding any data in their user record with the values from their contact record. Any fields enabled on the Contact Fields tab will be loaded.

When to use it: This operation would be used when you’ve just enabled a new WordPress field for sync with a custom field in your CRM that already contained data. This operation will load that custom field data into WordPress so it can then be displayed with a shortcode or used by other plugins.

Troubleshooting

The background worker is a complex process. It’s designed to work across all hosting environments, without affecting the speed or stability of your site.

While it’s generally very reliable, we have encountered issues with both caching and security plugins. The most common issue is that it will process a single batch of records (20 or so) and then not proceed to the next batch until the page is reloaded.

To aid in troubleshooting the background worker, WP Fusion does a status check on it every 5 seconds, and outputs some status information to the JavaScript console in your browser.

If you’re having issues with exporting data the simplest solution is to contact our support, and we can provide some suggestions based on your host and the plugins you have installed.

But if you’re inclined to troubleshoot it yourself, some things to note are:

  • Most servers have a max_execution_time of 30 seconds for PHP scripts. To get around this, the background worker will attempt to restart itself when the total_time value exceeds 20 (seconds). However, if the time_last_step value is greater than 10 (seconds), this could mean that the 30 second max_execution_time gets exceeded and the process times out.This most commonly happens on slower hosts when exporting WooCommerce orders, using the Enhanced Ecommerce addon. The process of creating a contact record, applying tags, creating products, registering an invoice, and adding the products to the invoice can take a long time for each order. You can speed up this process by first running an Export Users operation before exporting your orders. Since the contact records will already exist in your CRM, this reduces the time required to process each order.
  • WP Fusion tries to detect the available memory on your host and won’t let the background worker exceed 80% of available memory. However sometimes it’s not able to properly detect the available memory, and so memory_percent shows at 100% and the process prevents itself from running. You can disable this check by returning false from the wpf_batch_memory_exceeded filter.
  • Because WP Fusion makes sustained requests to admin-ajax.php over a long period of time, it can sometimes get blocked by security plugins. If you notice the batch status isn’t updating, it’s possible the background request is getting blocked. To see if that’s happening you can check your site’s access logs for any traffic to /wp-admin/admin-ajax.php?action=wpf_background_process is being blocked with a 403 (Unauthorized) status. If you’re using WordFence you can also put the plugin into Learning Mode before starting the batch operation to train it to allow this kind of traffic.

Was this helpful?