Import Stages

Every Super Speedy Imports run is a pipeline of independent stages. Each stage does one well-defined thing and writes its results to per-import staging tables that the next stage reads. The architecture matters because nearly every advanced feature — running just one stage, parallel workers, the import-mode presets, re-running after a failure — works at the stage boundary.

This page explains what each stage does. Once you know the vocabulary, the rest of the KB makes sense.

The big picture

A typical import flows like this:

load-csv → import-taxonomies → match-existing → update-posts / insert-posts
                                              → update-postmeta / insert-postmeta
                                              → upsert-relationships
                                              → image stages
                                              → fix-attributes (WC only)
                                              → process-deletes (if enabled)

Each stage runs once per import. The orchestrator (wp ssi <id> or the admin Run button) figures out the order from each stage’s priority (lower = earlier), spawns parallel workers where allowed, and writes per-stage timing + counts to the import history panel.

The stages

load-csv (priority 10)

Reads your CSV file into a per-import batch table (wp_super_speedy_imports_batch_<id>). Each CSV row becomes one row in the batch table with one column per mapped field. Hierarchical taxonomy values get parsed into a separate hierarchies aux table; flat taxonomy values go into a terms aux table.

This is the only stage that reads the CSV. Every later stage works against the staging tables.

Runs in parallel — by default 8 workers each chew through a portion of the CSV.

import-taxonomies (priority 20)

Creates any new WordPress terms your CSV references (categories, tags, attributes, custom taxonomies). Calls wp_insert_term for terms that don’t exist yet; resolves their term_taxonomy_id for everything in the batch.

Runs as a single worker — as of 2.55.8 this stage is hard-forced to 1 worker regardless of the configured count. Parallelising it measured only ~7% faster and risked a duplicate-key race in wp_insert_term; the stage now also recovers from that race via term_exists() instead of aborting, so it stays single-threaded by policy rather than out of necessity.

match-existing (priority 30)

For every row in the batch, looks for an existing WordPress post that matches by your configured unique identifier (typically _sku postmeta). When it finds a match, writes that post’s ID back into batch.post_id so downstream stages know whether to UPDATE or INSERT.

The single most important stage for re-imports: without it, every row would be inserted as a new post on every run.

update-posts (priority 40)

For batch rows where match-existing found an existing post (post_id is populated), this stage UPDATEs wp_posts.post_title, post_content, post_excerpt, etc. with the new CSV values.

Skipped by the Only insert new items import mode preset.

insert-posts (priority 50)

For batch rows where match-existing found nothing (post_id is NULL), this stage INSERTs new rows into wp_posts and writes the new IDs back into batch.post_id.

Skipped by the Only update existing items and Update prices / postmeta only import mode presets.

update-postmeta (priority 60)

For matched rows, UPDATEs wp_postmeta with the values from your post_meta config block (_sku, _regular_price, custom meta keys, etc.).

This is where price/stock changes actually land. Cheap to run in isolation — the Update prices / postmeta only preset uses just this plus load-csv and match-existing.

insert-postmeta (priority 70)

Same idea as update-postmeta but for newly inserted posts. Writes the initial postmeta rows for items that came into existence in insert-posts.

upsert-relationships (priority 80)

For every imported post, writes the right rows into wp_term_relationships so the post is associated with the categories, tags, brands, and attributes the CSV said it should be. Deletes any pre-existing relationships in the same taxonomies first (so a re-import correctly REPLACES the term set rather than appending).

This is usually the longest stage on bigger imports because the JOIN spans the whole batch table × the hierarchies/terms tables. The 2.47 covering indexes and 2.51 dedup fix specifically targeted this stage.

Since 2.55.9 the assign_all_hierarchy_levels option (“Assign every category level”) controls how many relationship rows get written per hierarchical post: when ON, a product is assigned EVERY level of a category path (parent + leaf), not just the leaf term; when OFF it gets only the leaf. New imports default this ON (pre-2.55.9 imports stay leaf-only), so expect the per-post relationship count — and the “Term Relationships” total — to be higher than the old leaf-only behaviour.

attach-existing-images (priority 90+ on post / product templates)

For featured images whose URL is already present in SSI’s image-lookup table (wp_ssi_image_lookup), sets the post’s _thumbnail_id to that attachment. No re-upload.

Runs in parallel — each worker handles a slice of the batch.

Changed in 2.55: the lookup used to read from a wp_postmeta row with meta_key = 'original_image_url'. As of 2.55 the canonical store is the dedicated wp_ssi_image_lookup table (indexed by url_hash for near-constant lookups regardless of URL length). The migration is automatic on first import after upgrade.

upload-remote-images (priority 90+ on post / product templates)

For featured images whose URL is NOT in the lookup table, downloads the remote file, runs it through wp_handle_upload to put it in wp-content/uploads/, creates the attachment post, and sets it as the post’s thumbnail. Also inserts a row into wp_ssi_image_lookup so future imports take the existing-attachment path.

Runs in parallel; each unique URL is uploaded by exactly one worker (the CTE-based owner assignment guarantees no duplicate uploads).

attach-gallery-images (priority 90+ on product templates)

Same as upload-remote-images but for the product’s gallery (the _product_image_gallery postmeta on WooCommerce products).

fix-attributes (priority 90+, WooCommerce only)

Rewrites the _product_attributes postmeta blob and rebuilds WooCommerce’s attribute lookup table for every imported product. Skipped on non-WooCommerce post types.

process-deletes (priority 90+)

If you’ve ticked “Delete missing items” in Additional Options, this stage deletes any post that was imported by a previous run of this same import but is missing from the current CSV. Force-delete or move-to-trash based on the linked option.

Also the ONLY stage that runs when you pass --delete-all on the CLI.

Why the stage system matters

You can rerun a single stage

If upload-remote-images failed halfway through but everything else succeeded:

wp ssi 5 upload-remote-images

Picks up where the previous run left off. The batch table is still populated; the stage just identifies rows that don’t yet have an attached thumbnail and processes those. Saves hours.

You can restrict the pipeline to a subset

The Import mode dropdown in Additional Options (since 2.53) lets you save a preset that only runs the stages you need. See the dedicated KB article for the full list of presets and their stage sets.

Per-stage timing is visible

The import-history panel (and the CLI summary at the end of every run) shows each stage’s wall-clock + CPU time. When an import gets slower, you can see WHICH stage got slower. See “Diagnosing slow imports” in the Server & Performance section.

Parallel vs sequential is explicit

Only four stages run in parallel: load-csv, attach-existing-images, upload-remote-images, fix-attributes. Everything else runs in a single process. This is by design — stages that write to global state (terms, auto-increment IDs, etc.) need coordination, while stages that operate per-row scale linearly with worker count.

Per-import staging tables

Each import has its own set of tables suffixed with the import’s ID — e.g. wp_super_speedy_imports_batch_5, wp_super_speedy_imports_terms_5, wp_super_speedy_imports_hierarchies_5. These persist between runs on purpose: when an import dies mid-stage, the staging tables let you re-run just the failing stage against the same data without re-loading the CSV. They also make post-mortem diagnostics possible (e.g. SELECT COUNT(*) FROM wp_super_speedy_imports_batch_5 WHERE post_id IS NULL tells you how many rows match-existing didn’t find).

load-csv drops and recreates these tables at the start of every full run, so re-importing the same CSV starts from a clean slate automatically.

What’s next

• Quick start — importing products: the practical day-1 walkthrough.
• Stage selection presets: how to control which stages actually run.
• Diagnosing slow imports: how to read the per-stage timing output.
• Stage system reference (developer KB): hooks and filters for extending the pipeline.