-
Notifications
You must be signed in to change notification settings - Fork 683
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Response format from Yahoo seems to have changed I keep getting this error. #952
Comments
As work around you may try to use solution from https://pypi.org/project/yfinance/ |
They started to encrypt the stores, looks like they encrypted with AES or something like that and now stores values comes as a string, not sure how to solve that |
Is there some way that I can implement a temporary solution using yfinance as an override and pull request it ? |
|
Oh yeah I have used this but I meant fixing pandas-datareader as I have already submitted the code for my dissertation so I can't change my code directly, the only way I can change my code is by changing the library hehe. sorry for confusion |
What are the stores? I will try anything to fix this as I have just submitted my interim dissertation and without this library, my code has no data :/ due to the nature of academia I dont think asking if the marker can override with yfinance will work |
Then you are the best candidate to fix this library. |
With a stroke of magical programmer luck I have managed to migrate some code from yfinance where now the data is decrypted and I have tested on a small number of stocks I am able to get their data.
It took me all day as I have never worked on such a professional code base so far. pandas-datareader was checkedout locally to my IDE and I have made all the changes there, what do you suggest me to do? I am sure there are still things needed to be added to the codebase like requirements etc? idk this is my first time. Thanks for any help. |
I have found a fix, as stated in above comment. Do you know how about I will go creating a pull request? I just tried on my IDE and got permission denied? Any suggestions? |
@raphi6 have you tried branching from the downloaded repository and then pushing your changes? It should say there's no upstream branch, by setting one you'll create a pull request. |
Ahh yes branching, I always forget. So now I will checkout pandas-datareader, create a branch and add my changes, create a pull request. And that should work? |
Yes I think so, it may not be the best way of doing it but I do not currently know of a better one. By setting an upstream branch and pushing your branch to it, it should create the PR. |
I just tried creating a new branch and commiting, pushing, then creating a PR and get the above, What did you mean by setting an upstream branch? |
You may be trying to push directly to master, which only the repo owners will have permission for (although it is advised never to push directly to master in any case). To not take up replies in the thread, I would recommend looking into the process of opening a PR on GitHub for another repo using online resources: Apologies I can't be any more help, quite new to Git in terms of contributing to 3rd party repos. |
I've opened a pull request with working code, be sure to check it out if it works for you guys. |
Hi, @raphi6 , I can confirm it works perfectly fine. Thanks a lot for the quick fix! Let's see if they approve the pull request. For reference, I cloned your pull request and installed it using the following sequence of commands (in Ubuntu 20.04):
The |
Can this be installed or implemented on Google Colabs? |
The following seems to work well on Google Colab.
|
|
Hello - I've got a few complex scripts running in jupiter notebook pulling data from yahoo finance - what are the exact commands which need to be entered in order to regain access so that I dont get the "string indices must be integers" error ? |
@pyproper Yes, it is. You can use this:
Notice I am using the commit hash here instead of a branch name, because it is |
Thanks very much, all looks good now! |
Perfect! Thank You |
I had to use pip3 to install, but now have the following error: Traceback (most recent call last): Thoughts? |
@KryptoEmman Have you tried |
This fix by @raphi6 needs to be made a priority. Yahoo API is bricked through PDR. |
Thank you! if anyone could get into contact with someone that can accept the PR, that would be great! |
Thanks very much everyone! I've migrated to yfinance and am looking to unplug/replug the data now from yfinance into my originally designed scripts. The only problem I seem to be having now relates to Dates: the format i want is simply year-month-date but I'm also getting a timestamp. I tried the formats above to no avail. Using widgets, some of the errors I've been encountering have been:
I'm using widgets for tickers and timeframes: any input would be much appreciated! many thanks |
stock1 = yf.download("ticker1", start="startDate", end="endDate") |
Or specifically, the date I'm getting in my dataframe is: 2022-02-17 00:00:00-05:00 Would anyone know how to get rid of the time component ? All I need is the date 2022-02-17 and not the 00:00:00-05:00 component I think that's why I'm getting getting a "TypeError: float() argument must be a string or a number, not 'Timestamp' |
After you get the data, you need to change the index, it's not interpreted as a date from yfinance. I had to convert mine to Pandas DatetimeIndex.
This does not get rid of the time component, but will allow you to filter by date. |
Im going on record and saying all who use Yahoo! Finance data need to put effort into maintaining Yfinance. This issue has been apparent for over a month and very little has been done by the team who manages this repo. Yfinance admin fixed it within a few days of the Yahoo! API encryption. |
Thanks very much for your help, pls excuse my lack of programing proficiency - I feel like I'm on the verge of getting this all done but just not quite there yet. I'm using: ticker1 = widgets.Text( ...this all works so far however I'm still getting Timestamp data and have not been able to get a string or number as the rest of my code would need. Where would I insert the above two lines you highlighted above ? I've been trying to convert the date index |
Interpreted conflict summary & progress breakdown:Appears Yahoo Finance (YF) has migrated their codebase to a different architecture--perhaps react+ Proposed crux:React-Redux architecture operates as a real-time, 'global state bus' to hold all variables/objects/states/values/facts/truths, and updates them without requiring end-user intervention between updates [1]. Long term scalability is the motivation here for YF. Proposed solution:
Sample YF backend key-value attribute handling for new_j data action event to update HistoricalPriceStore (all YF backend):events: [action.event, ...state.events] Sample reference case statement REDUCER for react-redux (typ. JSX syntax rather than javascript):case 'DECYRPTION_SUCCESS': Disclaimer: I am not an expert on this subject. Hope this helps your dissertation success. All for OP and OP for all! let Figure 1. Redux dataflow https://redux.js.org/tutorials/fundamentals/part-2-concepts-data-flow [1] |
|
Here I share a few tricks to solve this problem: import pandas then df = pdr.get_data_yahoo("TSLA", start="yyyy-mm-dd", end="yyyy-mm-dd") Hopefully it can be useful, thanks :) |
Thanks very much!! ... I'm using the yfin.pdr_override() but still one little issue however: Stock1 = pdr.get_data_yahoo("ticker1", start="yyyy-mm-dd", end="yyyy-mm-dd") now the error I'm getting is:
btw when I type in: Stock1.index.dtype --> datetime64[ns, America/Toronto] |
Thanks again everyone for your help, unfortunately I will be away all of next week but hopefully when I'm back and can give another crack at this (given all your input) I'll finally get things to work again (fingers crossed). Once again, thx very much in advance. Back in a week! |
This issue still up, but im very impressed with your tryes to solve it |
yahoo finance has new problem again. df = web.DataReader("SPY", data_source='yahoo', start='2023-01-01', end='2023-01-13', session = sesh) Error msg Traceback (most recent call last): File "C:\Users\peter\AppData\Local\Temp/ipykernel_27516/630144412.py", line 1, in File "C:\Users\peter\anaconda3\lib\site-packages\pandas\util_decorators.py", line 207, in wrapper File "C:\Users\peter\anaconda3\lib\site-packages\pandas_datareader\data.py", line 370, in DataReader File "C:\Users\peter\anaconda3\lib\site-packages\pandas_datareader\base.py", line 253, in read File "C:\Users\peter\anaconda3\lib\site-packages\pandas_datareader\yahoo\daily.py", line 227, in _read_one_data File "C:\Users\peter\anaconda3\lib\site-packages\pandas_datareader\yahoo\daily.py", line 81, in decrypt_cryptojs_aes File "C:\Users\peter\anaconda3\lib\site-packages\Crypto\Util\Padding.py", line 92, in unpad ValueError: Padding is incorrect. Thanks! |
Thank you @nmaiorana |
@nichaelwichterle1 - try this after creation of data frame: df['Date_New'] = data.index |
Thank you so much, after much trying I'm finally getting an output and all works for Stock1 and for that I'm overjoyed! My last question however is that I need to do the same process for Stock2, however the output for Stock2 seems to be a float. How can I convert Stock2 to a dt format like Stock1 so that I can run the same process and then finally graph both 1 and 2 ? |
Right, I am using yfinance to pull the data. I just am not able to change the date spacing on the x axis however. Anyone ? |
@nichaelwichterle1 what data type is the index of your underlying dataframe? If it is not import datetime as dt |
Thanks very much for your input. I was initially having problems because downloading the dataframe initially yielded year-month-day-hour-seconds which was problematic. I had to convert it via: this worked well enough but I have no idea how to change the date spacing on the x axis. |
when I code yfinance as follows by santosh1383 commented on Jan 20 import pandas df = pdr.get_data_yahoo("TSLA", start="yyyy-mm-dd", end="yyyy-mm-dd") I get exception following exception error. Can anyone tell me why and what am I missing? |
Solved my problem: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) GabrieleTap commented on Jul 19, 2021 pip uninstall yfinance |
from pandas_datareader import data as web yf.pdr_override() symbol = "000660.KS" start = datetime(2016,3,1) |
if you find an solution kindly update an solution @ sanjayram1515 |
File "/Users/yeison/miniforge3/envs/tf-metal-0.6.0/lib/python3.10/site-packages/pandas_datareader/yahoo/daily.py", line 153, in _read_one_data
data = j["context"]["dispatcher"]["stores"]["HistoricalPriceStore"]
TypeError: string indices must be integers
The text was updated successfully, but these errors were encountered: