Skip to content
All articles

What to check before collecting from a site

Verify the page address and necessary fields, compare its response with your expectations, and choose a suitable loading mode.

Akmelo Editorial TeamProduct and data collection
A website page and its inspection controls on a light backgroundImage unavailable

A page opens in your browser, so can you start collecting it? Check first. The source must provide the data under the same conditions your scenario will use. A product card may vary by region, selected variant, or content loaded later.

Take a few familiar product pages and write down the values you expect. This makes the setup easier to verify before processing the whole catalog.

Be specific about the page

Begin with a particular product page, catalog category, or search result URL. A store's home page says little about the structure of its other pages. Include several examples, especially if products come from different categories.

List the output fields precisely. Instead of “everything,” choose SKU, name, current price, and link, for example. If region or size matters, include it in the test conditions.

Also confirm that you may obtain and use the data. Being able to open a page does not, by itself, define the terms of later use. Respect source rules and acceptable request rates.

Inspect the response you receive

Akmelo can check a single page using the source site's network settings. This checks its content; it does not collect the full catalog. Compare the retrieved page with what you see when opening it normally.

HTTP success does not prove the page contains products. A source might return empty results, a message, or a different screen. Find at least one expected card and its fields in the response.

If ordinary HTTP lacks the values but they appear after JavaScript runs, check the page in browser mode. Compare both responses before choosing the scenario mode.

Check the boundaries

Find how to move to the next page and how to tell that the catalog has ended. Confirm that your scenario neither loops over the same result nor silently stops after the first page.

A small selection is enough for the first test. If it contains a gap, find its cause before expanding.

If the source responds differently than expected, save the URL and investigate before running again. The site guide explains the main settings. After that, move on to your first collection.