A number of users have contacted us about data usage warnings from Rithmic. Here's an example:
"
Rithmic's history plant guardian has observed that your user id, xxxx@hotmail.com, has requested historical market data resulting in 5063609137 bytes transmitted to your computers since Rithmic's systems most recent scheduled reboot. I am sure you can appreciate that large numbers of requests by large numbers of user ids can affect the flow of market data to all of Rithmic's users. Kindly review your programs and their use to make sure that such requests are intentional and needed. If they are not needed or they are not intentional, please change your programs and their use to reduce the number of requests that they make.
Further attempts to request or subscribe historical market data can cause our system to automatically suspend your access to the history plant."
In this case the 5063609137 is approximately 5gb.
On opening a Depth & Sales, Jigsaw will download the Level 1 history for the current session. Right now (as volumes are up) on Rithmic side, that's about 650mb for a session - although most people wouldn't be downloading that much data because they open the tool prior to the US open or just after. In these cases the history download is far less.
After a disconnect, or if you close and re-open Jigsaw, it will request the Level 1 history again. This request is made so that we can populate the Volume profile. It's been this way since 2012. Some vendors give us access to 1 range or 1 tick bars which also allows you to build a volume profile but using far less bandwidth.
To resolve this, there's some changes in progress at both Rithmic and Jigsaw. On Jigsaw side.
- Jigsaw - We'll shortly release a version that reduces the load by about 50% - because we get the L1 History in recon tape too.
- Jigsaw - We are implementing caching as part of our charting package which means any history will be downloaded only once from the servers and then from the cache. The charting package is currently going through performance improvements prior to release, so this change is not imminent.
- Rithmic - Are testing a new feed type that gives us 1 tick data feed. This will give us the ability to download VP but will have a lighter footprint.
The first item is done and is in QA, along with a number of other fixes. Rithmic will let us know when the 1 tick history is tested on their side and it'll take us a day or 2 to implement that. Thus far Rithmic have not cut anyone off and these are just warnings.
We are working with Rithmic on this, so please bear with us.