Learn how you can become a Python programmer in just 12 weeks.

    We respect your privacy. Unsubscribe at anytime.

    Fix get_data_yahoo from Pandas Datareader

    What will we cover?

    If you use get_data_yahoo from Pandas Datareader and it suddenly stopped working, then we will look at how to fix.

    UPDATE: We will also cover a FIX of the current issue in 2023

    Current 2023 ISSUE and FIX

    If you get an error that ends with.

    TypeError: string indices must be integers
    

    You get that because there has been breaking changes at Yahoo! Finance.

    An easy workaround is as follows.

    Install yfinance.

    pip install yfinance
    

    Then this code will work (NOTICE the .data in the import from pandas_datareader.data).

    import pandas_datareader.data as pdr
    import yfinance as yf
    yf.pdr_override()
    from datetime import datetime
    data = pdr.get_data_yahoo('^GSPC', datetime(1970, 1, 1))
    

    The above code should work.

    Below you have the full error trace.

    TypeError                                 Traceback (most recent call last)
    /var/folders/5f/vgbh9pkd4wzf4ysrpf9wsr800000gn/T/ipykernel_82918/2899696086.py in <module>
    ----> 1 data = pdr.get_data_yahoo('AAPL', datetime(2022, 1, 1))
    ~/opt/anaconda3/lib/python3.9/site-packages/pandas_datareader/data.py in get_data_yahoo(*args, **kwargs)
         78 
         79 def get_data_yahoo(*args, **kwargs):
    ---> 80     return YahooDailyReader(*args, **kwargs).read()
         81 
         82 
    ~/opt/anaconda3/lib/python3.9/site-packages/pandas_datareader/base.py in read(self)
        251         # If a single symbol, (e.g., 'GOOG')
        252         if isinstance(self.symbols, (string_types, int)):
    --> 253             df = self._read_one_data(self.url, params=self._get_params(self.symbols))
        254         # Or multiple symbols, (e.g., ['GOOG', 'AAPL', 'MSFT'])
        255         elif isinstance(self.symbols, DataFrame):
    ~/opt/anaconda3/lib/python3.9/site-packages/pandas_datareader/yahoo/daily.py in _read_one_data(self, url, params)
        151         try:
        152             j = json.loads(re.search(ptrn, resp.text, re.DOTALL).group(1))
    --> 153             data = j["context"]["dispatcher"]["stores"]["HistoricalPriceStore"]
        154         except KeyError:
        155             msg = "No data fetched for symbol {} using {}"
    TypeError: string indices must be integers
    

    The Error and Problem (old problem)

    Consider this code.

    import pandas_datareader as pdr
    from datetime import datetime
    data = pdr.get_data_yahoo('^GSPC', datetime(1970, 1, 1))
    

    It has been working up until now. But suddenly it writes.

    Traceback (most recent call last):
      File "/Users/rune/PycharmProjects/TEST/test_yahoo.py", line 4, in <module>
        data = pdr.get_data_yahoo('^GSPC', datetime(1970, 1, 1))
      File "/Users/rune/PycharmProjects/TEST/venv/lib/python3.8/site-packages/pandas_datareader/data.py", line 86, in get_data_yahoo
        return YahooDailyReader(*args, **kwargs).read()
      File "/Users/rune/PycharmProjects/TEST/venv/lib/python3.8/site-packages/pandas_datareader/base.py", line 253, in read
        df = self._read_one_data(self.url, params=self._get_params(self.symbols))
      File "/Users/rune/PycharmProjects/TEST/venv/lib/python3.8/site-packages/pandas_datareader/yahoo/daily.py", line 153, in _read_one_data
        resp = self._get_response(url, params=params)
      File "/Users/rune/PycharmProjects/TEST/venv/lib/python3.8/site-packages/pandas_datareader/base.py", line 181, in _get_response
        raise RemoteDataError(msg)
    pandas_datareader._utils.RemoteDataError: Unable to read URL: https://finance.yahoo.com/quote/^GSPC/history?period1=10800&period2=1627523999&interval=1d&frequency=1d&filter=history
    Response Text:
    b'<!DOCTYPE html>\n  <html lang="en-us"><head>\n  <meta http-equiv="content-type" content="text/html; charset=UTF-8">\n      <meta charset="utf-8">\n      <title>Yahoo</title>\n      <meta name="viewport" content="width=device-width,initial-scale=1,minimal-ui">\n      <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">\n      <style>\n  html {\n      height: 100%;\n  }\n  body {\n      background: #fafafc url(https://s.yimg.com/nn/img/sad-panda-201402200631.png) 50% 50%;\n      background-size: cover;\n      height: 100%;\n      text-align: center;\n      font: 300 18px "helvetica neue", helvetica, verdana, tahoma, arial, sans-serif;\n  }\n  table {\n      height: 100%;\n      width: 100%;\n      table-layout: fixed;\n      border-collapse: collapse;\n      border-spacing: 0;\n      border: none;\n  }\n  h1 {\n      font-size: 42px;\n      font-weight: 400;\n      color: #400090;\n  }\n  p {\n      color: #1A1A1A;\n  }\n  #message-1 {\n      font-weight: bold;\n      margin: 0;\n  }\n  #message-2 {\n      display: inline-block;\n      *display: inline;\n      zoom: 1;\n      max-width: 17em;\n      _width: 17em;\n  }\n      </style>\n  <script>\n    document.write(\'<img src="//geo.yahoo.com/b?s=1197757129&t=\'+new Date().getTime()+\'&src=aws&err_url=\'+encodeURIComponent(document.URL)+\'&err=%<pssc>&test=\'+encodeURIComponent(\'%<{Bucket}cqh[:200]>\')+\'" width="0px" height="0px"/>\');var beacon = new Image();beacon.src="//bcn.fp.yahoo.com/p?s=1197757129&t="+new Date().getTime()+"&src=aws&err_url="+encodeURIComponent(document.URL)+"&err=%<pssc>&test="+encodeURIComponent(\'%<{Bucket}cqh[:200]>\');\n  </script>\n  </head>\n  <body>\n  <!-- status code : 404 -->\n  <!-- Not Found on Server -->\n  <table>\n  <tbody><tr>\n      <td>\n      <img src="https://s.yimg.com/rz/p/yahoo_frontpage_en-US_s_f_p_205x58_frontpage.png" alt="Yahoo Logo">\n      <h1 style="margin-top:20px;">Will be right back...</h1>\n      <p id="message-1">Thank you for your patience.</p>\n      <p id="message-2">Our engineers are working quickly to resolve the issue.</p>\n      </td>\n  </tr>\n  </tbody></table>\n  </body></html>'
    

    What to do?

    The fix (old fix)

    There has been a breaking change and you need to update your Pandas Datareader.

    You can upgrade to the newest version as follows.

    pip install --upgrade pandas-datareader
    

    It should update it to version 0.10.0 or later.

    Then the code should work again.

    Python for Finance: Unlock Financial Freedom and Build Your Dream Life

    Discover the key to financial freedom and secure your dream life with Python for Finance!

    Say goodbye to financial anxiety and embrace a future filled with confidence and success. If you’re tired of struggling to pay bills and longing for a life of leisure, it’s time to take action.

    Imagine breaking free from that dead-end job and opening doors to endless opportunities. With Python for Finance, you can acquire the invaluable skill of financial analysis that will revolutionize your life.

    Make informed investment decisions, unlock the secrets of business financial performance, and maximize your money like never before. Gain the knowledge sought after by companies worldwide and become an indispensable asset in today’s competitive market.

    Don’t let your dreams slip away. Master Python for Finance and pave your way to a profitable and fulfilling career. Start building the future you deserve today!

    Python for Finance a 21 hours course that teaches investing with Python.

    Learn pandas, NumPy, Matplotlib for Financial Analysis & learn how to Automate Value Investing.

    “Excellent course for anyone trying to learn coding and investing.” – Lorenzo B.

    22 thoughts on “Fix get_data_yahoo from Pandas Datareader”

    1. While using my python script that uses pandas-DataReader v10 & pandas v1.3.3 get_data_yahoo(), when it first starts off it will show the last 5 days correctly because I am tailing this for the last 5 days, then after about a minute or two it changes to what you see below, I am using python 3.7.10 The date moves back 2 weeks and the volume from the last 4 days is off a lot, the current day is always correct ok

      I run my script on linux using the “watch -n1

      Code below
      —————————————–
      def get_data(tickers):
      df = pdr.get_data_yahoo(tickers)
      return df

      for df in tickers:
      df = get_data(df) # this function retrieve’s stock information from yahoo finance
      df = pd.DataFrame(df.Close)
      print(df.tail(5)
      ——————————————–

      #1 — works correctly Close
      Date Close
      2021-08-15 11.2500
      2021-09-16 9.9600
      2021-09-17 11.0600
      2021-09-20 11.7300
      2021-09-21 13.7001

      #2 — doesn’t work correctly Close
      Date Close
      2021-08-31 5.9600
      2021-09-01 7.4000
      2021-09-02 6.4900
      2021-09-03 6.7000
      2021-09-21 13.7001

      how can anyone help me

      Reply
      • Hi Frank,

        It looks like you have a list of Tickers you iterate over. You get the data for all of the and print the last retrieved data.

        The line taking Close in a DataFrame is maybe not needed?

        What is the goal here? For each iteration see the last five days of the specific ticket?

        If you share the list of tickers you are looking at and the full code I can take a look at it.

        Reply
    2. I am getting information for one stock and that is the closing price and volume and I am using pandas-datareader get_data_yahoo() function, when I first run the stock one time it looks like the first photo, but what I am doing is using the “watch -n1 ” to get updates every 1 to 5 seconds so I get live updates, but after some time the data coming back goes back to September 09 2021 – August 31, 2021 like the second photo and the volume numbers don’t make any sense they are going into the billions, I have tried everything, different version of python3 + different versions of pip3 and pandas and pandas-datareader, the question is why does it keep going back to the last two weeks and why is my volume messed up, except for the current day., it only took 6 minutes for it to change to photo two

      https://photos.app.goo.gl/Yh7kCXUJQWq1UXKx8

      Reply
      • I think this will do what you want:


        import pandas_datareader as pdr
        from datetime import datetime, timedelta

        while True:
        df = pdr.get_data_yahoo('AAPL', datetime.utcnow() - timedelta(days=7))
        print(df)

        Reply
      • Hi Bob,
        Just make sure we have installed in the right environment.

        Try to execute:
        import pandas_datareader as pdr
        print(pdr.__version__)

        It should output: 0.10.0

        If not, it is not installed (upgraded) in the right environment.

        What environment are you using?

        Reply
    3. I am using Python 2.7 because it is what the author of an instruction video to scrape finance data from yahoo uses but I guess I’m gonna have to upgrade to Python 3 because I only have pandas 0.8.1.

      Reply
    4. I am using Spyder. I tried updating pandas but it said I already had the latest version so I guess 0.8.1 is the latest version for Python 2.7.

      Reply
    5. Hi, how can I check the version of pandas-datareader? I am using Anaconda distribution. After struggling for hours, I finally figured out how to install pandas-datareader (need to use conda install -c anaconda pandas-datareader in Anaconda Prompt).

      However I still get the “Unable to read URL” error. I tried to check the datareader version using “print(dpr.__version__) but it says that there is no attribute ‘__version__’. Please help.

      Reply
      • Hi Alex,

        If you run this:

        import pandas_datareader as pdr
        pdr.__version__

        In a cell and it does give you no attribute error, then you can run the following in a cell:

        !pip freeze
        Then you need to find the line with

        pandas-datareader==0.10.0

        It is ordered, so you should be able to locate it. (my output gives it for version 0.10.0, your’s might be different).

        Reply
    6. Hello Rune, I’m unable to get multiple tickers. It keeps giving me this error

      ValueError Traceback (most recent call last)
      /tmp/ipykernel_67023/2049989489.py in
      2 start = dt.datetime(2022, 1, 1)
      3
      —-> 4 data = pdr.get_data_yahoo(tickers, start)

      /usr/local/lib/python3.8/dist-packages/pandas_datareader/data.py in get_data_yahoo(*args, **kwargs)
      78
      79 def get_data_yahoo(*args, **kwargs):
      —> 80 return YahooDailyReader(*args, **kwargs).read()
      81
      82

      /usr/local/lib/python3.8/dist-packages/pandas_datareader/base.py in read(self)
      256 df = self._dl_mult_symbols(self.symbols.index)
      257 else:
      –> 258 df = self._dl_mult_symbols(self.symbols)
      259 return df
      260

      /usr/local/lib/python3.8/dist-packages/pandas_datareader/base.py in _dl_mult_symbols(self, symbols)
      283 stocks[sym] = df_na
      284 if PANDAS_0230:
      –> 285 result = concat(stocks, sort=True).unstack(level=0)
      286 else:
      287 result = concat(stocks).unstack(level=0)

      /usr/local/lib/python3.8/dist-packages/pandas/core/frame.py in unstack(self, level, fill_value)
      8411 from pandas.core.reshape.reshape import unstack
      8412
      -> 8413 result = unstack(self, level, fill_value)
      8414
      8415 return result.__finalize__(self, method=”unstack”)

      /usr/local/lib/python3.8/dist-packages/pandas/core/reshape/reshape.py in unstack(obj, level, fill_value)
      476 if isinstance(obj, DataFrame):
      477 if isinstance(obj.index, MultiIndex):
      –> 478 return _unstack_frame(obj, level, fill_value=fill_value)
      479 else:
      480 return obj.T.stack(dropna=False)

      /usr/local/lib/python3.8/dist-packages/pandas/core/reshape/reshape.py in _unstack_frame(obj, level, fill_value)
      499 def _unstack_frame(obj, level, fill_value=None):
      500 if not obj._can_fast_transpose:
      –> 501 unstacker = _Unstacker(obj.index, level=level)
      502 mgr = obj._mgr.unstack(unstacker, fill_value=fill_value)
      503 return obj._constructor(mgr)

      /usr/local/lib/python3.8/dist-packages/pandas/core/reshape/reshape.py in __init__(self, index, level, constructor)
      138 )
      139
      –> 140 self._make_selectors()
      141
      142 @cache_readonly

      /usr/local/lib/python3.8/dist-packages/pandas/core/reshape/reshape.py in _make_selectors(self)
      190
      191 if mask.sum() 192 raise ValueError(“Index contains duplicate entries, cannot reshape”)
      193
      194 self.group_index = comp_index

      ValueError: Index contains duplicate entries, cannot reshape

      Reply
    7. i have the same error :
      ticker = [‘AAPL’]
      start = dt.datetime(2020, 1, 1)
      data = pdr.get_data_yahoo(ticker, start)
      print(data)
      ValueError: Index contains duplicate entries, cannot reshape

      Reply
      • It should work.
        Make sure you have the right formatting.

        import pandas_datareader as pdr
        import datetime as dt

        ticker = ["AAPL"]
        start = dt.datetime(2020, 1, 1)
        data = pdr.get_data_yahoo(ticker, start)
        print(data)

        If this does not work – make sure you have the right version of pandas datareader.

        Reply
    8. Hi Rune , i have the same problem

      tickers = [‘AAPL’ ‘MSFT’]
      start = dt.datetime(2020, 1, 1)
      data = pdr.get_data_yahoo(tickers, start)

      print(data)

      i i insert tickers ‘ValueError: Index contains duplicate entries, cannot reshape?

      but if in tickers i insert ‘GBPUSD=X’, ‘EURGBP=X’ this is ok

      Reply
    9. The following code was working until a Yesterday:

      from pandas_datareader import data as pdr
      stockticker = pdr.get_data_yahoo(‘GOOG’, ‘2022-12-19’)

      Now errors out for the past two days with:

      Traceback (most recent call last):
      File “USP.py”, line 77, in
      stockticker = pdr.get_data_yahoo(‘GOOG’, ‘2022-12-19’)
      File “/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pandas_datareader/data.py”, line 80, in get_data_yahoo
      return YahooDailyReader(*args, **kwargs).read()
      File “/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pandas_datareader/base.py”, line 253, in read
      df = self._read_one_data(self.url, params=self._get_params(self.symbols))
      File “/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pandas_datareader/yahoo/daily.py”, line 153, in _read_one_data
      data = j[“context”][“dispatcher”][“stores”][“HistoricalPriceStore”]
      TypeError: string indices must be integers, not ‘str’

      Was and still running pandas-datareader v0.10.0 (latest version available) and Python v3.11 (upgraded to v3.11.1, same issue)

      Thoughts?
      Thanks!

      Reply
    10. import numpy as np
      import pandas as pd

      import matplotlib.pyplot as plt
      from matplotlib import rcParams

      import pandas_datareader.data as pdr
      import yfinance as yf
      yf.pdr_override()
      from datetime import datetime

      import seaborn as sb
      sb.set()

      file_loc = “Desktop/Misc/stocksdata.xlsx”
      df = pd.read_excel(file_loc, sheet_name=’Sheet1′, usecols=”A,B”)

      for x in range(600,900):
      stock_symbol=df.iloc[x][0]
      stock_name=df.iloc[x][1]
      #Extrating last 5 prices of stock under consideration
      Rel=pdr.get_data_yahoo(stock_symbol,”)
      Rel_close=pd.DataFrame(Rel.Close)
      Rel[‘MA21’]=Rel_close.Close.rolling(21).mean()
      Rel[‘Std’]=Rel[‘Close’].std()
      Rel[‘Range1’]=Rel[‘MA21’]+2.5*Rel[‘Std’]
      Rel[‘Range2’]=Rel[‘MA21’]-2.5*Rel[‘Std’]

      closeprice=Rel.loc[ datetime(2023,5,5),’Close’]
      Range1price=Rel.loc[ datetime(2023,5,5),’Range1′]
      Range2price=Rel.loc[ datetime(2023,5,5),’Range2′]
      print(stock_symbol)
      if (closeprice > Range1price or closeprice < Range2price):
      print(stock_name)

      Getting this error

      1 Failed download:
      – GRAPHITE.NS: ValueError("time data '' does not match format '%Y-%m-%d'")

      Reply

    Leave a Comment