Hi As i mentioned in an earlier email, this is extremely likely to be yahoo making life difficult for web scrapers of their data. GNC will have to catch up to what it is doing for a “permanent” solution.
It looks to be very much the same problem where various errors (402, 500 etc) are returned by that same server. A temporary solution that MAY work if using a VPN is to set it to appear like USA (say NY which did work for a while or SanFran and sometime other countries around Europe and Asia). It seems to be random. But the problem is linked to the login sessions not remaining valid (but I’m not qualified to say exactly how). The previous post also pointed to yahooquery which is a python webscraper module and similar problems with yfinance (another similar module). You can track it down in GitHub or the developer’s website. Anyway, if I’m right this won’t go away and won’t fix itself. Regards James > On 28 Mar 2024, at 12:43 pm, Kalpesh Patel <kalpesh.pa...@usa.net> wrote: > > Looking at "ERROR 429: Too Many Requests." I am guessing that they are > throttling your queries because number of requests from IP address you have > exceeds what they believe to be a normal person browsing would be making... > > -----Original Message----- > From: David G. Pickett <davidgpick...@comcast.net> > Sent: Wednesday, March 27, 2024 3:12 PM > To: gnucash-user@gnucash.org > Subject: Re: [GNC] Finance Quotes Yahoo off the rails again? > > URLs are scraped off, so here I am on a plain text email client with more > info on the bug. It looks like something on the web does not like some > clients: > > From quote log: > > * AMEX:DXJ Finance::Quote reported failure with error: Error retrieving quote > for DXJ. Attempt to fetch the URL > https://query2.finance.yahoo.com/v11/finance/quoteSummary/?symbol=DXJ&modules=price,summaryDetail,defaultKeyStatistics > resulted in HTTP response 500 (Internal Server Error) > > Trying with wget: > > dgp@dgp-p6803w:~ > $ wget > 'https://query2.finance.yahoo.com/v11/finance/quoteSummary/?symbol=DXJ&modules=price,summaryDetail,defaultKeyStatistics' > > | timex wc > --2024-03-26 14:18:10-- > https://query2.finance.yahoo.com/v11/finance/quoteSummary/?symbol=DXJ&modules=price,summaryDetail,defaultKeyStatistics > Resolving query2.finance.yahoo.com (query2.finance.yahoo.com)... > 69.147.82.60, 69.147.82.61 > Connecting to query2.finance.yahoo.com > (query2.finance.yahoo.com)|69.147.82.60|:443... connected. > HTTP request sent, awaiting response... 429 Too Many Requests > 2024-03-26 14:18:25 ERROR 429: Too Many Requests. > > 0 0 0 > real 15.11 > user 0.00 > sys 0.00 > dgp@dgp-p6803w:~ > $ wget > 'https://query2.finance.yahoo.com/v11/finance/quoteSummary/?symbol=DXJ&modules=price,summaryDetail,defaultKeyStatistics' > > | timex wc > --2024-03-26 14:19:34-- > https://query2.finance.yahoo.com/v11/finance/quoteSummary/?symbol=DXJ&modules=price,summaryDetail,defaultKeyStatistics > Resolving query2.finance.yahoo.com (query2.finance.yahoo.com)... > 69.147.82.60, 69.147.82.61 > Connecting to query2.finance.yahoo.com > (query2.finance.yahoo.com)|69.147.82.60|:443... connected. > HTTP request sent, awaiting response... 429 Too Many Requests > 2024-03-26 14:19:50 ERROR 429: Too Many Requests. > > 0 0 0 > real 15.17 > user 0.00 > sys 0.00 > dgp@dgp-p6803w:~ > $ > > With Google Chrome, get: > > https://query2.finance.yahoo.com/v11/finance/quoteSummary/?symbol=DXJ&modules=price,summaryDetail,defaultKeyStatistics > > {"quoteSummary":{"result":[{"summaryDetail":{"maxAge":1,"priceHint":{"raw":2,"fmt":"2","longFmt":"2"},"previousClose":{"raw":107.84,"fmt":"107.84"},"open":{"raw":108.56,"fmt":"108.56"},"dayLow":{"raw":108.38,"fmt":"108.38"},"dayHigh":{"raw":108.88,"fmt":"108.88"},"regularMarketPreviousClose":{"raw":107.84,"fmt":"107.84"},"regularMarketOpen":{"raw":108.56,"fmt":"108.56"},"regularMarketDayLow":{"raw":108.38,"fmt":"108.38"},"regularMarketDayHigh":{"raw":108.88,"fmt":"108.88"},"dividendRate":{},"dividendYield":{},"exDividendDate":{},"payoutRatio":{},"fiveYearAvgDividendYield":{},"beta":{},"trailingPE":{"raw":12.077986,"fmt":"12.08"},"forwardPE":{},"volume":{"raw":515869,"fmt":"515.87k","longFmt":"515,869"},"regularMarketVolume":{"raw":515869,"fmt":"515.87k","longFmt":"515,869"},"averageVolume":{"raw":1097601,"fmt":"1.1M","longFmt":"1,097,601"},"averageVolume10days":{"raw":1277770,"fmt":"1.28M","longFmt":"1,277,770"},"averageDailyVolume10Day":{"raw":1277770,"fmt":"1.28M","longFmt":"1,277, > 770"},"bid":{"raw":108.82,"fmt":"108.82"},"ask":{"raw":108.83,"fmt":"108.83"},"bidSize":{"raw":1200,"fmt":"1.2k","longFmt":"1,200"},"askSize":{"raw":900,"fmt":"900","longFmt":"900"},"marketCap":{},"yield":{"raw":0.0293,"fmt":"2.93%"},"ytdReturn":{},"totalAssets":{"raw":4416737792,"fmt":"4.42B","longFmt":"4,416,737,792"},"expireDate":{},"strikePrice":{},"openInterest":{},"fiftyTwoWeekLow":{"raw":68.18,"fmt":"68.18"},"fiftyTwoWeekHigh":{"raw":109.28,"fmt":"109.28"},"priceToSalesTrailing12Months":{},"fiftyDayAverage":{"raw":100.636,"fmt":"100.64"},"twoHundredDayAverage":{"raw":90.06975,"fmt":"90.07"},"trailingAnnualDividendRate":{"raw":0.0,"fmt":"0.00"},"trailingAnnualDividendYield":{"raw":0.0,"fmt":"0.00%"},"navPrice":{"raw":107.95631,"fmt":"107.96"},"currency":"USD","fromCurrency":null,"lastMarket":null,"volume24Hr":{},"volumeAllCurrencies":{},"circulatingSupply":{},"algorithm":null,"maxSupply":{},"startDate":{},"tradeable":false},"defaultKeyStatistics":{"maxAge":1,"priceHint":{"raw" > :2,"fmt":"2","longFmt":"2"},"enterpriseValue":{},"forwardPE":{},"profitMargins":{},"morningStarOverallRating":{},"morningStarRiskRating":{},"category":"Japan > Stock","bookValue":{},"priceToBook":{},"annualReportExpenseRatio":{},"ytdReturn":{"raw":0.22916919,"fmt":"22.92%"},"beta3Year":{"raw":0.43,"fmt":"0.43"},"totalAssets":{"raw":4416737792,"fmt":"4.42B","longFmt":"4,416,737,792"},"yield":{"raw":0.0293,"fmt":"2.93%"},"fundFamily":"WisdomTree","fundInceptionDate":{"raw":1150416000,"fmt":"2006-06-16"},"legalType":"Exchange > Traded > Fund","threeYearAverageReturn":{"raw":0.24652371,"fmt":"24.65%"},"fiveYearAverageReturn":{"raw":0.2016703,"fmt":"20.17%"},"priceToSalesTrailing12Months":{},"lastFiscalYearEnd":{},"nextFiscalYearEnd":{},"mostRecentQuarter":{},"earningsQuarterlyGrowth":{},"revenueQuarterlyGrowth":{},"netIncomeToCommon":{},"trailingEps":{},"forwardEps":{},"pegRatio":{},"lastSplitFactor":null,"lastSplitDate":{},"enterpriseToRevenue":{},"enterpriseToEbitda":{},"52WeekChange":{},"SandP52WeekChange":{},"lastDividendValue":{},"lastCapGain":{},"annualHoldingsTurnover":{}},"price":{"maxAge":1,"preMarketChangePercent":{"raw":0.010200354,"fmt":"1.02%"},"preMarketChange":{"raw":1.1000061,"fmt":"1.10"},"preMarketTime":1711456034,"preMarketPrice":{"raw":108.94,"fmt":"108.94"},"preMarketSource":"DELAYED","postMarketChange":{},"postMarketPrice":{},"regularMarketChangePercent":{"raw":0.009319408,"fmt":"0.93%"},"regularMarketChange":{"raw":1.0050049,"fmt":"1.01"},"regularMarketTime":1711477138,"priceHint":{"raw":2,"fmt > ":"2","longFmt":"2"},"regularMarketPrice":{"raw":108.845,"fmt":"108.85"},"regularMarketDayHigh":{"raw":108.88,"fmt":"108.88"},"regularMarketDayLow":{"raw":108.38,"fmt":"108.38"},"regularMarketVolume":{"raw":515869,"fmt":"515.87k","longFmt":"515,869.00"},"averageDailyVolume10Day":{"raw":1277770,"fmt":"1.28M","longFmt":"1,277,770"},"averageDailyVolume3Month":{"raw":1097601,"fmt":"1.1M","longFmt":"1,097,601"},"regularMarketPreviousClose":{"raw":107.84,"fmt":"107.84"},"regularMarketSource":"FREE_REALTIME","regularMarketOpen":{"raw":108.56,"fmt":"108.56"},"strikePrice":{},"openInterest":{},"exchange":"PCX","exchangeName":"NYSEArca","exchangeDataDelayedBy":0,"marketState":"REGULAR","quoteType":"ETF","symbol":"DXJ","underlyingSymbol":null,"shortName":"WisdomTree > Japan Hedged Equity ","longName":"WisdomTree Japan Hedged Equity > Fund","currency":"USD","quoteSourceName":"Nasdaq Real Time > Price","currencySymbol":"$","fromCurrency":null,"lastMarket":null,"volume24Hr":{},"volumeAllCurrencies":{},"circulatingSupply":{},"marketCap":{}}}],"error":null}} > > > Last night cron script still failing: > > flatpak run --command=gnucash-cli org.gnucash.GnuCash --quotes get -- > /home/dgp/GnuCash/ubuntu-DGPickett.gnucash > F: X11 socket /tmp/.X11-unix/X0 does not exist in filesystem, trying to use > abst ract socket instead. > > ** (process:211848): WARNING **: 11:28:45.396: Failed to connect to bus: > Could not connect: Connection refused > Found Finance::Quote version 1.58. > * 12:33:54 WARN <gnc.price-quotes> [GncQuotesImpl::parse_one_quote()] > Skipped NYSE:T - Finance::Quote returned fetch failure. > Reason Error retrieving quote for T. Attempt to fetch the URL > https://query2.finance.yahoo.com/v11/finance/quoteSummary/?symbol=T&modules=price,summaryDetail,defaultKeyStatistics > resulted in HTTP response 500 (Internal Server Error) > * 12:33:54 WARN <gnc.price-quotes> [GncQuotesImpl::parse_one_quote()] > Skipped FUND:WFILX - Finance::Quote returned fetch failure. > Reason Error retrieving quote for WFILX. Attempt to fetch the URL > https://query2.finance.yahoo.com/v11/finance/quoteSummary/?symbol=WFILX&modules=price,summaryDetail,defaultKeyStatistics > resulted in HTTP response 500 (Internal Server Error) > * 12:33:54 WARN <gnc.price-quotes> [GncQuotesImpl::parse_one_quote()] > Skipped FUND:FDCAX - Finance::Quote returned fetch failure. > . > > . > > . > > . > > > _______________________________________________ > gnucash-user mailing list > gnucash-user@gnucash.org > To update your subscription preferences or to unsubscribe: > https://lists.gnucash.org/mailman/listinfo/gnucash-user > ----- > Please remember to CC this list on all your replies. > You can do this by using Reply-To-List or Reply-All. _______________________________________________ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user ----- Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All.
