How does the batch import interface determine if the article already exists, and will it write twice if it is executed repeatedly?

How to judge if the article already exists and whether it will be written twice when calling the document content import interface for batch import? AnQiCMS provides the /api/import/archive document content import interface, which supports querying whether the article already exists by title. The idempotent approach is to solidify the 'first check then write' order: for each title, check for existence first, skip or go to the update branch if it already exists, and write after confirming that it does not exist. Following this order, re-execution will not write the same article twice.

The order of 'first check then write'

Perform two steps for each article during the import loop. The first step is to call the existence query interface with the title to get the determination of 'exists/does not exist'; the second step is to proceed with creation if it does not exist, or choose to skip, rename and create again, or transfer to the update process if it already exists. Do not underestimate this order; scripts written directly without any consideration are most likely to produce duplicate entries when interrupted and restarted, and post-event cleanup is much more costly than pre-event query.

The boundary of title-based duplicate checking

There are two types of boundary cases that need to be predefined for the title-based key:

• The source data already has articles with the same title (in different columns), and it is necessary to add a column or date suffix to distinguish, otherwise the second one will always be judged as existing and skipped;
• During the revision, what should be updated:

The query duplicate check only solves the problem of

How to record the cursor for running continuously from a breakpoint?

There are thousands of articles that cannot be run at once, and it should be possible to resume after interruption. The method is to maintain a cursor locally: after completing one, append the title and processing result (create/skip/fail) to the log file; when restarting, read the cursor first, skip the completed items in the log directly, and then complement it with a search by title. The significance of two-level insurance lies in: the local cursor is fast, the interface query is accurate, and if the cursor file is lost or the machine is changed, the existence checked by title can still ensure that there is no repeated writing.

Which end to limit the high-frequency call first

When speeding up batch scripts, it is easy to overwhelm the write. The high-frequency capabilities of AnQiCMS interface calls are concentrated on content aspects such as document lists and details, categories and tags, image upload, and comment replies, among which read calls can be moderately concurrent, while import/archive write endpoints should control the pace: single concurrent or low concurrency in sequence, with failure retry and exponential backoff. It is more cost-effective to limit the write end first, rather than release both ends at the same time and then deal with duplicate data. Specific parameters and authorization methods can be found in the category of interface documents within the station, and calls should be made according to the document specifications.

How to check the import results

The run does not equal the import, check three sets of accounts at the end:

1. Number of entries—the sum of successful imports, skipped entries, and failed entries should equal the total number of entries in the source list;
2. Status—check the actual status of some entries in the background (official document, draft, or pending release) and ensure it matches the script's expectations;
3. Content—randomly check a few articles to ensure that the title, category, tags, and custom fields are complete, and confirm that the image content has accessible images.

The failure list should be kept separately, and only the failed parts should be re-run after the cause of the failure is fixed; the entire process should be coordinated with batch import (ZIP compressed package and Excel table are suitable for one-time library construction, interface is suitable for programmatic continuous running) according to the data form selected channel.

Common questions

The interface returns that it already exists, but I can't find it in the background?First, search accurately by title and check the recycle bin, deleted entries still occupy the title judgment; clean up thoroughly before importing when confirming to rebuild.

Can multiple import processes be run concurrently?It is not recommended, concurrent writing at the write end is the main source of duplicate entries and lock contention; to speed up, prioritize parallelizing the query and preparation stages (reading files, cleaning data), while keeping writing as a single stream.

What if the cursor file is damaged after interruption?Just delete the cursor and run it from scratch, searching for existence by title will hold the entries already written, in the worst case, it will take an extra round of query time, but no duplicate content will be produced.