Troubleshooting Data Transfer
History Read Failures due to OPC Server Points Limits
History reads can fail when the amount of requested data exceeds the points limit of the OPC Server.
To avoid the problem, locate the History Transfer Controller object which is responsible for the data transfer, and reduce the amount of data requested in each acquisition frame. In the Transporter Configuration > Acquisition Options Compound reduce the value for the Acquisition Frame or the Basket Limit.
Only in the rare case that the History Transporter is working independently, i.e. without a managing History Transfer Controller, these changes need to be made directly in the Acquisition Options Compound of the History Transporter object itself.
| Do not make Configuration changes directly on the History Transporter objects if it is managed by a History Transfer Controller! Only independent History Transporter objects should be configured directly! |
Filling History Gaps
For detailed information on how to fill history gaps, depending on the object type controlling the original history transfer process, see Filling History Gaps for History Transfer Controllers.
FAQs
General
Q: What is the processing order of the History Transporter?
A: Context > Continuous > History > Snapshot
Q: Can the VQT data received by the History Transporter be modified?
A: Yes, this can be done using the Postprocess Callback property. Please use caution when modifying data using this property.
Q: How can I restart the history processing from the beginning?
A: Enable Reset Status by selecting the checkbox and click Apply to save the changes.
Q: Why is there no context data when the History Transporter is used with the gRPC Datasource?
A: Context extraction is not currently supported for this datasource.
Q: I have done one backfill, how do I start another?
A: Change the Depth property to reflect the new backfill range and select the checkbox for Reset Status, then click Apply to save the changes.
Q: Is it possible to get aggregated data from the source system?
A: No, at present, this is not supported.
Q: Why does my History Transporter appear stuck?
A: Check the Heartbeat property. If the latest update of this property is less than the currenttime() - recurrence, then the History Transporter may be stuck. Please contact Support for assistance.
Q: What are possible HIST Status messages?
A: There are several potential status messages:
-
BACKLOG - the tag has been processed during the latest cycle and some points have been received for this tag
-
NO DATA - the tag was processed during the latest cycle but no data points was received for this tag
-
TIMEOUT - the tag was not processed during the latest cycle due to a timeout (Slicing)
-
POINT LIMIT - the tag was not processed during the latest cycle as the point limit was reached (Hedging)
-
PIMS LIMIT - the tag was not processed during the latest cycle as the point limit on the source side was reached
-
OUT OF RANGE - the tag was not processed during the cycle because its range has already been processed. It will be processed when the Acquisition Frame moves to the next range.
-
ERROR - this tag had an error in the latest cycle
-
DELAYED - the priority of the tag was decreased due to a previous error on this tag
Q: My History Transporter does not process history data but UTD data is there. Why does this occur?
A: One potential reason for this issue is that the History Transporter does not have enough time to process both Continuous and Historical Data. If the Continuous data uses all the available time, the History Transporter will not be able to process History Data.
Check the logs and Process State property. If this is the cause of the issue, then information indicating this problem should be found here.
To resolve this problem, try increasing the cycle duration, increase the Hedging and Slicing limits, or decrease the number of tags in the Tag Configuration Table.
Q: What does Refetch in the Tag Configuration Table do?
A: The Refetch property is only used by the History Transfer Controller and so, it does not impact the Tag Configuration Table.
Tags
Q: How does Context Call affect historical processing?
A: Context Call is used to filter out invalid tags so that time is not wasted on them.
When Transfer Context is disabled, the Data Availability will be 100%. Context errors will be shown in the Configuration Problems table.
Q: What is the difference between History Progress and Data Availability?
A: History Progress shows the progression of historical transfer but only for tags which went through Context Call.
Data Availability shows how many tags went through Context Call.
Q: Why do some tags not have expected data despite the History Transporter being in a good state?
A: Check the status of these tags in the State Table. In this table, look for the CTX Status, HIST Status and CONT Status columns (dependent on enabled functionality of History Transporter). The tag may have an error message in the corresponding ERROR column.
Q: How can I re-register tags which have been filtered out by context call?
A: Enable Reload by selecting the checkbox and click Apply to save the changes.
Q: Can I reset individual tags?
A: At present, resetting of individual tags is not supported.
Q: Can the History Transporter create tags outside of itself?
A: No, a History Transporter cannot create tags outside of itself. If tags should be backfilled, these tags must be created before starting the History Transporter.
Q: How is history delivered to the tags?
A: The History Transporter processes the same Acquisition Frame for each of the configured tags, then it moves to the next Acquistion Frame. Tags with the lowest progress are always prioritised.
If a tag returns an error, its priority will be lowered. If there is time remaining at the end of the cycle, the History Transporter will try to process failed tags again.
Q: What is the maximum number of tags that can be processed by a single History Transporter?
A: There is currently no limit but it is not recommended to have more than 50,000 tags for a single History Transporter.
The performance of a History Transporter is dependent on the system on which it runs. Increasing the number of tags is likely to affect the performance of the History Transporter.
Performance
Q: How do I know that the History Transporter is running at full capacity?
A: Check the Process State property. For maximum performance, the History Transporter should always be in Processing Mode.
Q: What could cause my History Transporter to slow down significantly at some point?
A: At some point, the History Transporter may have experienced a time period with very high data density. To check how many data points are received by the History Transporter per cycle, look at the Points Diagnostics.
Alternatively, the issue may be on the Server side. In the log, you should be able to find the Information Logs with Message Cycle End.
Q: How many History Transporters can be created under a single Node?
A: There is currently no limit but it is not recommended to have more than 20-30 on a single Node.
It is important to note that each History Transporter is a dedicated thread Lua script. Too many dedicated scripts may be problematic for the Node.
In addition, configuring the Invocation Delay to distribute the History Transporters in such a manner that they do not fight for hardware resources is very difficult.
Q: How much RAM does a History Transporter use?
A: The memory usage of a History Transporter is dependent on the number of tags, the amount of VQTs for each of these tags and the data type.
It is normal to see large variations in memory usage for the History Transporter. Usually, the History Transporter will use significantly more RAM during its processing because it receives a large amount of VQTs. When the VQTs are processed, the RAM usage should decrease as most of the memory will be released.