Troubleshooting Source Collector Runs

Overview

When a Source Collector does not run as expected, the cause is not always obvious. A missed-run notification can mean the collector never started, that it started but encountered an error, or even that it ran successfully but found no data to collect.

This guide helps you identify the problem, investigate the cause, and decide what to do next.

Escalation

Not every issue requires contacting Varibill. Most problems can be investigated and resolved by following the steps in this guide. When they cannot, use the following escalation path:

1. Investigate the issue yourself using the steps in this guide. Many common problems, such as expired credentials, missing data at the source, configuration errors etc. can be resolved directly.
2. Contact your internal technical team if the issue involves infrastructure you manage, such as a self-hosted collector server, network connectivity, firewall rules, or access to a source system. Your internal technical team has direct access to servers and systems that Varibill cannot reach.
3. Contact Varibill if you have ruled out all of the above. See When to Contact Varibill Support for specific scenarios.

Understanding Missed Runs vs Failures

A “missed run” and a “failed run” are not the same thing. Understanding the difference will save you time when investigating.

Missed Run

A missed run means that no data batch was recorded within the expected schedule window. Varibill checks your collector’s CRON schedule against the date of its most recent batch. If a batch should have been created but was not, the run is counted as missed.

A missed run can happen for several reasons:

• The collector never started (infrastructure or scheduling issue)
• The collector started but encountered an error before submitting data
• The collector ran successfully but found no records to collect

That last point is important: a collector that runs and collects zero records does not create a batch. From Varibill’s perspective, this looks identical to a run that never happened. See Scenario B: Collector Ran Successfully but No Data Was Collected.

Failed Run

A failed run means the collector started but could not complete. This produces a Failed execution status (visible for Cloud-hosted collectors) and usually leaves behind console logs with an error message explaining what went wrong.

When a Missed Run Resolves

When a collector runs successfully and submits data, its missed-run counter resets to zero automatically. You will not receive a separate “recovered” notification. The missed-run emails simply stop.

What You Can See Depends on How Your Collector Is Hosted

Not all collectors provide the same level of diagnostic visibility in the Varibill UI. The information available to you depends on where the collector runs.

Cloud-hosted collectors give you the most visibility. Everything runs in Varibill’s infrastructure, so execution history, logs, and data files are all accessible from the Source Collector Editor.

Self-hosted collectors run on your own servers. Varibill can see batch results and record counts, but execution history and log files are only available on the host machine itself. If you cannot access the server directly, your internal technical team should be your first point of contact. They will have access to the infrastructure where the collector runs.

Scenario A: Collector Failed to Run or Encountered an Error

Use this section if:

• You received a missed-run notification email
• The Dashboard shows the collector under “Failed Collectors” with a missed run count
• You suspect the collector did not run at all, or that it started but failed to execute or collect data

Step 1: Check the Dashboard

1. Go to the Dashboard (home page).
2. Select the Failed Collectors tab.
3. Find your collector in the list and note the Missed Runs count.

A missed runs count above zero means Varibill expected data from one or more scheduled runs but did not receive it.

Step 2: Check Job Execution History (Varibill Cloud Only)

1. Open the collector in the Source Collector Editor.
2. Scroll to the Job Execution section.
3. Look at the most recent executions:

Step 3: Check Console Logs (Varibill Cloud Only)

Console logs capture the full output from the collector’s run, including error messages.

1. In the Source Collector Editor, go to Job Execution Files.
2. Navigate to the console/ directory.
3. Open the log file that corresponds to the failed run’s time period.
4. Look for lines beginning with [ERROR]. These indicate what went wrong.

See Common Errors and What They Mean for help interpreting specific error messages.

Step 3 (Self-Hosted): Check Logs on Your Host Machine

If your collector is self-hosted, log files exist on the machine where the collector runs, not in Varibill.

• Console output: Check the Source Collector Server logs, or run docker logs for the relevant container.
• API response logs: Check the log/ directory within the collector’s working folder.
• Pending data files: Check the unrated/ or rated/ directory for .rdy files (see Step 5).

If you do not have access to the server, contact your internal technical team. They can check the logs on your behalf and report back on what they find.

Step 4: Check API Response Logs (Varibill Cloud Only)

API response logs record the result of each data submission to Varibill.

1. In Job Execution Files, navigate to the log/ directory.
2. Open the .json files to see the API response for each submission.
3. Look for non-success status codes (anything other than 200 or 201).

Step 5: Check for Pending Uploads

When a collector successfully collects data but fails to upload it, the data is saved as .rdy files. These files are automatically retried on the next scheduled run.

Varibill Cloud: In Job Execution Files, navigate to the unrated/ or rated/ directory and look for files ending in .rdy.

Self-hosted: Check the same directories on the host machine.

If .rdy files are present, the data has been collected but not yet delivered to Varibill. In most cases, the next scheduled run will pick them up and complete the upload automatically. No manual action is needed.

Step 6: Check Batch History

1. In the Source Collector Editor, scroll to the Batches section.
2. Look for batches with an Unsuccessful status. This means the data was submitted but failed validation.
3. Review the Feedback column for details on what went wrong during validation.

A batch status of Successful means the data was received and validated. Unsuccessful usually indicates a data quality issue rather than a collector failure.

Scenario B: Collector Ran Successfully but No Data Was Collected

Use this section if:

• The Job Execution history shows a Succeeded status, but no new batch appeared
• The console logs show a clean start-to-finish run with no errors
• The Dashboard still counts a missed run

This is one of the most common support scenarios. The collector ran as expected, but there was simply no new data to collect from the source system.

Why This Happens

Source Collectors retrieve data from external systems (APIs, databases, cloud platforms). If the source system has no new data for the collection period, the collector completes normally but does not create a batch in Varibill. Because there is no batch, the system records it as a missed run.

Common reasons for no data at the source:

• The external service had no new usage, transactions, or records for the period
• A change was made in the source system that stopped data from being generated (for example, a subscription was cancelled, an integration was paused, or API permissions were changed)
• The source system’s data retention has expired and older records are no longer available

What to Check

1. Verify the source system directly. Log in to the external platform or service your collector retrieves data from and confirm that data exists for the expected period.
2. Check whether anything changed. Was a subscription cancelled? Were API credentials rotated? Was a service paused?
3. Review Source Data. Go to Billing Data > Source Data and filter by your collector and date range. If no records appear for the expected period, the collector did not find data to submit.

What to Do

If the source system genuinely has no data, no action is needed. The collector is working correctly. The missed-run count will reset automatically when the next run successfully collects and submits data.

If you believe data should exist but the collector is not finding it, check whether:

• The collector’s connection credentials are still valid
• The source system’s API or data format has changed
• Any filters or parameters on the collector need updating

If the issue involves access to the source system or changes on the vendor’s side, contact your internal team or the source system provider. If you have ruled out source-side issues and the collector still appears to be missing data, contact Varibill Support with the collector name and the date range in question.

Common Errors and What They Mean

These are errors you may encounter in console logs or as feedback on unsuccessful batches.

Connection and Authentication Errors

Data Submission Errors

Configuration Errors

Safe Actions You Can Take

Recovery Limitations

It is important to understand what can and cannot be recovered after a missed or failed run:

• Historical data recovery depends on the collector type. Most collectors only collect data that is currently available in the source system. However, Historically Capable collectors can retrieve past or date-range-based data directly from the source system. If your collector is Historically Capable, a manual run may be able to recover missed data without additional coordination.
• Source system data retention. If the source system has a retention policy and the data has been purged, it cannot be recovered through Varibill.
• Pending uploads resolve automatically. If .rdy files are present from a previous run, the next scheduled run will attempt to upload them.

When to Contact Varibill Support

Contact Varibill Support if any of the following apply:

When contacting Support, include:

• The Source Collector name
• The date and time of the missed or failed run
• Any error messages from the console logs (screenshot or copy-paste)
• Whether the collector is Cloud-hosted or Self-hosted

Batch Status Reference

When reviewing Batch History, the following statuses indicate where data is in the processing pipeline:

Related Articles

• Configuring a New Source Collector
• Self-Hosted Source Collectors
• Source Collector Classification and Definitions