Replies: 8 comments 31 replies
-
Your first code snippet shows you writing the daily prices. The second snippet shows you querying the merged prices. But are you calling write_merged_prices_for_contract to update the merged prices with the new daily prices? |
Beta Was this translation helpful? Give feedback.
-
Thank you Todd, I was following the same code as Rob's seed price script under sysinit. Let me give write merge price a try. Could you please explain high level the concept of merged prices? Merge of what? Is it the merged prices that are used in all cases, especially in building adjusted prices. |
Beta Was this translation helpful? Give feedback.
-
You need to do both: first write daily prices (and hourly if you wanted them). Then merge them. In Rob's script, it is done at line 46. Overall, it goes like this: Contract prices at specific frequencies --> merged contract prices (all frequencies) --> multiple prices --> adjusted prices. Just writing merged prices would have worked just now because you already had the daily prices. But your script needs to do both. |
Beta Was this translation helpful? Give feedback.
-
Run update_sampled_contracts. I usually just wait for that to run as part of my daily process, so I didn't think about it. |
Beta Was this translation helpful? Give feedback.
-
Hi guys, I'm having somewhat similar situation with SOYBEAN_mini but here a contract is missing. Last contract in existing multiple prices is 20231100 but 20240100 in seeded prices from IB. The problem is that seed didn't get 20231100 at all, there is no line for that in the seed log. and no such contract in db. So how do I resolve situation like this? In step 3, $ python3 $PYSYS_CODE/sysinit/futures/adhoc/update_provided_multiple_prices.py [33996 rows x 6 columns] [188 rows x 6 columns] Multiple prices rolled to 20231100 at 29/06/2023 And the created roll calendar, 2023-11-29 is the latest price I have: |
Beta Was this translation helpful? Give feedback.
-
" The problem is that seed didn't get 20231100 at all, there is no line for that in the seed log. and no such contract in db. So how do I resolve situation like this?" I have noticed a similar issue when trying to add new instruments. Sometimes it just doesn't download the right contract months and I get stuck with a bunch of contracts that are empty dataframes since they don't really exist. I haven't dug deep into why it fails for some instruments. What I usually do is just feed in a list of desired contact months into the seed_prices... code and usually that will work. |
Beta Was this translation helpful? Give feedback.
-
I got few instruments more now that IB seed succeeds but got stuck with GAS_US_mini. $ python3 $PYSYS_CODE/sysinit/futures/adhoc/update_provided_multiple_prices.py gives the following, ending up with assertion error. existing multiple prices (Rob's, presumably):
DATETIME new multiple prices (seeded from IB):
DATETIME So existing data has 202310 and 202311 in seeded data. Provided multiple prices has no data after 2023-08-31, so there is no roll from 202310 to 202311. Existing data has roll from 202309 to 202310 in 2023-08-11: Script's generated roll calendar: Tried to play with the 2023-08-11 but can't get it right. So how should the rolls look like in this situation? |
Beta Was this translation helpful? Give feedback.
-
I am at the initial stage of seeding prices from IB and building roll calendars. I am having a data issue with the roll calendars derived from multiple prices. There were no issues running Rob's script and the roll calendars now extend monotonically to the end of March 2024 using the develop branch multiple prices data repo. The problem is that there are two rolls in February 2023 on 2/9 and 2/20 and March is skipped. All the other rolls look correct. What do you do when this happens, just fix it manually I assume? It makes me nervous that it got it wrong. The rollconfig.csv for AEX is a simple one: AEX,FGHJKMNQUVXZ,-5,1,FGHJKMNQUVXZ,19. There are no anomalies in AEX multiple prices in February or March 2023. I cant see how we would end up with two rolls in February and none in March? I don't want to get bogged down thinking about it because there is a ton of work to get to a solid price library, but it does make me wonder and maybe someone has a simple explanation? Next I used the rollcalendars_from_arcticprices_to_csv.py to build a roll calendar with more recent data I fetched from IB. My simple idea is that I will append the multiple prices derived roll calendar with data from roll calendar derived from the more recent price IB history. I'll write a script to strip the last roll in the multiple prices derived roll calendars (as its just the last date in the price series and not a true roll), and append with up to date rolls. Lastly, why are there multiple roll calendars for the same futures contract in the shipped roll calendar csv's, ie. BITCOIN #1, BITCOIN #2, BITCOIN #3? Going through them, they appear to add/increment 1 month advances in current_contract, next_contract, and carry_contract all for the same DATE_TIME. What are these three variations used for and why do they have different incremental contract holds, next and carry? There are a few instances of multiple variations of the same contract in the roll calendar repo. |
Beta Was this translation helpful? Give feedback.
-
Hi Everyone,
Working on the latest master branch. It seems for some reason I have a data gap in BITCOIN instrument in 202206, so when I try to build the roll calendars and adjusted prices again, the process cannot go beyond this date.
I have written my own seed price script, because I only want to download historical daily prices from IB starting with 202206 contract (the reason being IB only allows you one year historical data, the default seed script in PST causes the price gap to be pushed to even earlier).
Some snippets from my seeding code
In debugger I can see that the June 2022 prices cover the entire month of June, and then there are no errors writing into arctic.
But then when I try to safely modify roll parameters from interactive control, the process gets stuck in June 2022 again, so I wrote another scripts to check what price data I have in arctic
I see no price data after June 3rd, even though I have just seeded the whole month of June's price data and beyond.
Any help greatly appreciated.
Best regards,
Emre
Beta Was this translation helpful? Give feedback.
All reactions