Contract not found

[mariekarpinska]

I'm getting the following error when running run_daily_fx_and_contract_updates:

Traceback (most recent call last):
File "C:\AC3 15 Vol Trading System\pysystemtrade\sysproduction\run_daily_fx_and_contract_updates.py", line 34, in
run_daily_fx_and_contract_updates()
File "C:\AC3 15 Vol Trading System\pysystemtrade\sysproduction\run_daily_fx_and_contract_updates.py", line 15, in run_daily_fx_and_contract_updates
price_process.run_process()
File "c:\ac3 15 vol trading system\pysystemtrade\syscontrol\run_process.py", line 106, in run_process
self._main_loop_over_methods()
File "c:\ac3 15 vol trading system\pysystemtrade\syscontrol\run_process.py", line 132, in _main_loop_over_methods
timer_class.check_and_run(**kwargs)
File "c:\ac3 15 vol trading system\pysystemtrade\syscontrol\timer_functions.py", line 124, in check_and_run
self.run_function(**kwargs)
File "c:\ac3 15 vol trading system\pysystemtrade\syscontrol\timer_functions.py", line 299, in run_function
self._function(**kwargs)
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\update_sampled_contracts.py", line 73, in update_sampled_contracts
update_active_contracts_with_data(data, instrument_code=instrument_code)
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\update_sampled_contracts.py", line 86, in update_active_contracts_with_data
update_active_contracts_for_instrument(instrument_code, data)
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\update_sampled_contracts.py", line 104, in update_active_contracts_for_instrument
check_key_contracts_have_not_expired(instrument_code=instrument_code, data=data)
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\update_sampled_contracts.py", line 418, in check_key_contracts_have_not_expired
list_of_expired_ids = [
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\update_sampled_contracts.py", line 419, in
has_contract_expired(
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\update_sampled_contracts.py", line 447, in has_contract_expired
actual_contract = data_contracts.get_contract_from_db(contract)
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\data\contracts.py", line 190, in get_contract_from_db
db_contract = self.get_contract_from_db_given_code_and_id(
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\data\contracts.py", line 200, in get_contract_from_db_given_code_and_id
contract_object = self.db_contract_data.get_contract_object(
File "c:\ac3 15 vol trading system\pysystemtrade\sysdata\futures\contracts.py", line 41, in get_contract_object
raise ContractNotFound(
syscore.exceptions.ContractNotFound: Contract AEX/20240200 not found

So similarly as to Roll Report Crashing #1332, I tried update_sampled_contracts, which gave me the folllowing:

Instrument code?(Return for ALL)AEX
Traceback (most recent call last):
File ".\pysystemtrade\sysproduction\update_sampled_contracts.py", line 453, in
update_sampled_contracts()
File ".\pysystemtrade\sysproduction\update_sampled_contracts.py", line 45, in update_sampled_contracts
update_contracts_object.update_sampled_contracts(
File ".\pysystemtrade\sysproduction\update_sampled_contracts.py", line 73, in update_sampled_contracts
update_active_contracts_with_data(data, instrument_code=instrument_code)
File ".\pysystemtrade\sysproduction\update_sampled_contracts.py", line 86, in update_active_contracts_with_data
update_active_contracts_for_instrument(instrument_code, data)
File ".\pysystemtrade\sysproduction\update_sampled_contracts.py", line 104, in update_active_contracts_for_instrument
check_key_contracts_have_not_expired(instrument_code=instrument_code, data=data)
File ".\pysystemtrade\sysproduction\update_sampled_contracts.py", line 418, in check_key_contracts_have_not_expired
list_of_expired_ids = [
File ".\pysystemtrade\sysproduction\update_sampled_contracts.py", line 419, in
has_contract_expired(
File ".\pysystemtrade\sysproduction\update_sampled_contracts.py", line 447, in has_contract_expired
actual_contract = data_contracts.get_contract_from_db(contract)
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\data\contracts.py", line 190, in get_contract_from_db
db_contract = self.get_contract_from_db_given_code_and_id(
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\data\contracts.py", line 200, in get_contract_from_db_given_code_and_id
contract_object = self.db_contract_data.get_contract_object(
File "c:\ac3 15 vol trading system\pysystemtrade\sysdata\futures\contracts.py", line 41, in get_contract_object
raise ContractNotFound(
syscore.exceptions.ContractNotFound: Contract AEX/20240200 not found

So I followed #1292, and tried rolling the contract and updating adjusted prices, but ran into the following:

Interactive Update Roll status:

How do you want to do your rolls today?
0: Manually input instrument codes and manually decide when to roll
1: Cycle through instrument codes automatically, but manually decide when to roll
2: Cycle through instrument codes automatically, auto decide when to roll, manually confirm rolls
3: Cycle through instrument codes automatically, auto decide when to roll, automatically roll

Your choice?
2024-07-18 22:18:14 DEBUG config {'type': 'config', 'stage': 'config'} Adding config defaults
Instrument code?(Return to EXIT)AEX
c:\ac3 15 vol trading system\pysystemtrade\sysobjects\futures_per_contract_prices.py:35: UserWarning: Pan
ndas doesn't allow columns to be created via a new attribute name - see https://pandas.pydata.org/pandas-docs/stable/indexing.html#attribute-access
self._as_df = price_data_as_df
2024-07-18 22:18:17 DEBUG Interactive_Update-Roll-Status Running report Roll report sysproduction.reporting.roll_report.roll_report console {'instrument_code': 'AEX', 'reporting_api': <sysproduction.reporting.api.reportingApi object at 0x0000021109FE5850>}
Traceback (most recent call last):
File "c:\ac3 15 vol trading system\pysystemtrade\syscore\cache.py", line 48, in _get_from_store
value = self.store[key]
KeyError: "_get_roll_data_dict/('AEX',)/{}"

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "c:\ac3 15 vol trading system\pysystemtrade\syscore\cache.py", line 29, in get
value_from_store = self._get_from_store(key)
File "c:\ac3 15 vol trading system\pysystemtrade\syscore\cache.py", line 50, in _get_from_store
raise missingData("Missing cache element %s" % key) from e
syscore.exceptions.missingData: Missing cache element _get_roll_data_dict/('AEX',)/{}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File ".\pysystemtrade\sysproduction\interactive_update_roll_status.py", line 953, in
interactive_update_roll_status()
File ".\pysystemtrade\sysproduction\interactive_update_roll_status.py", line 61, in interactive_update_roll_status
function_to_call(api=api, data=data)
File ".\pysystemtrade\sysproduction\interactive_update_roll_status.py", line 138, in update_roll_status_manual_cycle
manually_report_and_update_roll_state_for_code(
File ".\pysystemtrade\sysproduction\interactive_update_roll_status.py", line 557, in manually_report_and_update_roll_state_for_code
run_roll_report(api, instrument_code)
File ".\pysystemtrade\sysproduction\interactive_update_roll_status.py", line 610, in run_roll_report
report_results = run_report_with_data_blob(config, api.data)
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\reporting\reporting_functions.py", line 76, in run_report_with_data_blob
report_results = run_report_from_config(report_config=report_config, data=data)
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\reporting\reporting_functions.py", line 87, in run_report_from_config
report_results = report_function(data, **report_kwargs)
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\reporting\roll_report.py", line 40, in roll_report
formatted_output.append(reporting_api.table_of_roll_data(instrument_code))
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\reporting\api.py", line 562, in table_of_roll_data
result_pd = self._roll_data_as_pd(instrument_code)
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\reporting\api.py", line 569, in _roll_data_as_pd
roll_data_dict = self.roll_data_dict(instrument_code)
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\reporting\api.py", line 578, in roll_data_dict
return self.cache.get(self._get_roll_data_dict, instrument_code)
File "c:\ac3 15 vol trading system\pysystemtrade\syscore\cache.py", line 31, in get
value_from_store = self._calculate_and_store(
File "c:\ac3 15 vol trading system\pysystemtrade\syscore\cache.py", line 38, in _calculate_and_store
value = function_instance(*args, **kwargs)
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\reporting\api.py", line 589, in _get_roll_data_dict
roll_data = get_roll_data_for_instrument(instrument_code, data)
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\reporting\data\rolls.py", line 48, in get_roll_data_for_instrument
carry_expiry_days = c_data.days_until_carry_expiry(instrument_code)
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\data\contracts.py", line 72, in days_until_carry_expiry
carry_expiry = self.get_carry_expiry(instrument_code)
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\data\contracts.py", line 237, in get_carry_expiry
return self.get_actual_expiry(instrument_code, contract_id)
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\data\contracts.py", line 207, in get_actual_expiry
contract_object = self.get_contract_from_db_given_code_and_id(
File "c:\ac3 15 vol trading system\pysystemtrade\sysproduction\data\contracts.py", line 200, in get_contract_from_db_given_code_and_id
contract_object = self.db_contract_data.get_contract_object(
File "c:\ac3 15 vol trading system\pysystemtrade\sysdata\futures\contracts.py", line 41, in get_contract_object
raise ContractNotFound(
syscore.exceptions.ContractNotFound: Contract AEX/20240200 not found`

Updating adjusted prices seemed to work, but after rerunning all the other scripts, nothing changed:

py .\pysystemtrade\sysproduction\update_multiple_adjusted_prices_GPDedit.pyC:\AC3 15 Vol Trading System>
Configuring sim logging
C:\AC3 15 Vol Trading System\pysystemtrade\venv\3.8.10\lib\site-packages\arctic\store_pandas_ndarray_store.py:6: FutureWarning: The Panel class is removed from pandas. Accessing it from the top-level namespace will also be removed in the next version
from pandas import DataFrame, Series, Panel
2024-07-18 22:16:24 DEBUG config {'type': 'config', 'stage': 'config'} Adding config defaults
Instrument code?(Return for ALL)AEX
2024-07-18 22:16:27 DEBUG Update-Multiple-Adjusted-Prices {'instrument_code': 'AEX'} Updating log attributes
2024-07-18 22:16:27 CRITICAL Update-Multiple-Adjusted-Prices {'instrument_code': 'AEX'} Something went wrong with adjusted prices for AEX! [current_contract_dict() takes 1 positional argument but 2 were given]
Instrument code?(Return to EXIT)
(3.8.10) PS C:\AC3 15 Vol Trading System> py .\pysystemtrade\sysproduction\update_multiple_adjusted_prices.py
Configuring sim logging
C:\AC3 15 Vol Trading System\pysystemtrade\venv\3.8.10\lib\site-packages\arctic\store_pandas_ndarray_store.py:6: FutureWarning: The Panel class is removed from pandas. Accessing it from the top-level namespace will also be removed in the next version
from pandas import DataFrame, Series, Panel
2024-07-18 22:17:19 DEBUG config {'type': 'config', 'stage': 'config'} Adding config defaults
Instrument code?(Return for ALL)AEX
2024-07-18 22:17:22 DEBUG Update-Multiple-Adjusted-Prices {'instrument_code': 'AEX'} Updating log attributes
c:\ac3 15 vol trading system\pysystemtrade\sysobjects\futures_per_contract_prices.py:35: UserWarning: Pandas doesn't allow columns to be created via a new attribute name - see https://pandas.pydata.org/pandas-docs/stable/indexing.html#attribute-access
self._as_df = price_data_as_df
2024-07-18 22:17:22 INFO arctic.arctic Mongo Quota: arctic_production.futures_multiple_prices 0.144 / 10 GB used
2024-07-18 22:17:23 DEBUG Update-Multiple-Adjusted-Prices {'component': 'arcticFuturesMultiplePricesData', 'instrument_code': 'AEX'} Wrote 73257 lines of prices for AEX to Arctic connection: host mongodb://127.0.0.1:27018, db production, collection futures_multiple_prices
2024-07-18 22:17:23 INFO Update-Multiple-Adjusted-Prices {'instrument_code': 'AEX'} Added data for instrument AEX
2024-07-18 22:17:23 INFO arctic.arctic Mongo Quota: arctic_production.futures_adjusted_prices 0.059 / 10 GB used
2024-07-18 22:17:23 DEBUG Update-Multiple-Adjusted-Prices {'component': 'arcticFuturesAdjustedPricesData', 'instrument_code': 'AEX'} Wrote 73206 lines of prices for AEX to Arctic connection: host mongodb://127.0.0.1:27018, db production, collection futures_adjusted_prices
2024-07-18 22:17:23 INFO Update-Multiple-Adjusted-Prices {'component': 'arcticFuturesAdjustedPricesData', 'instrument_code': 'AEX'} Added data for instrument AEX

What am I missing? What can I do to resolve this?

[craigmediaservices]

Did you also try updating sampled contracts?

[mariekarpinska]

Yeah, that's the second section of output above

[craigmediaservices]

Sorry, I skimmed too quick.

Next I'd check - does your system think it has a position? That is dealt with in interactive_order_stack.

I think I had a similar issue and tried manually editing the database, but that caused more problems.

If you need to start from scratch use the develop branch and pull Carver's latest price CSVs, upload to parquet, and then pull interactive brokers latest prices.