Bug 22834 - The Yahoo finance API shutdown broke perl-finance-quote - an update is required to fix Gnucash's quote fetching.
Summary: The Yahoo finance API shutdown broke perl-finance-quote - an update is requir...
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: 6
Hardware: All Linux
Priority: Normal major
Target Milestone: ---
Assignee: QA Team
QA Contact:
URL: http://cpansearch.perl.org/src/ECOCOD...
Whiteboard: MGA6-64-OK
Keywords: advisory, validated_update
Depends on:
Blocks:
 
Reported: 2018-03-27 09:16 CEST by Edward d'Auvergne
Modified: 2018-06-06 20:16 CEST (History)
4 users (show)

See Also:
Source RPM: perl-Finance-Quote-1.380.0-3.mga6
CVE:
Status comment:


Attachments

Description Edward d'Auvergne 2018-03-27 09:16:50 CEST
Description of problem:

Yahoo shut down their finance API last year (2017), breaking perl-finance-quote.  In response, perl-finance-quote switched to AlphaVantage to remain functional, and then released version 1.39:

http://cpansearch.perl.org/src/ECOCODE/Finance-Quote-1.39/htdocs/index.html

This notably affects the quote fetching feature of Gnucash.


Version-Release number of selected component (if applicable):

All versions below 1.39.


How reproducible:

100%


Steps to Reproduce:

To see this issue:

1. Open Gnucash.
2. Create some accounts with different currencies.
3. Select "Tools->Price editor" from the menu.
4. Click on the "Get Quotes" button.

A pop up window will appear saying "Unable to retrieve quotes for these items:... CURRENCY:EUR, CURRENCY:GBP, CURRENCY:USD, etc.".
Edward d'Auvergne 2018-03-27 09:19:21 CEST

URL: (none) => http://cpansearch.perl.org/src/ECOCODE/Finance-Quote-1.39/htdocs/index.html

Comment 1 Marja Van Waes 2018-03-27 12:59:05 CEST
Assigning to the registered perl-Finance-Quote maintainer

Btw, why do we have perl-Finance-Quote-1.380.0-3.mga6 instead of perl-Finance-Quote-1.38.0-3.mga6? Is that because of 

  Version:    %perl_convert_version %{upstream_version}

If so, why does it need to be converted like that?

Source RPM: (none) => perl-Finance-Quote-1.380.0-3.mga6
CC: (none) => marja11
Assignee: bugsquad => shlomif

Comment 2 Shlomi Fish 2018-03-27 18:39:30 CEST
(In reply to Marja van Waes from comment #1)
> Assigning to the registered perl-Finance-Quote maintainer
> 

thanks! I submitted 1.470 to 6/updates_testing.

> Btw, why do we have perl-Finance-Quote-1.380.0-3.mga6 instead of
> perl-Finance-Quote-1.38.0-3.mga6? Is that because of 
> 
>   Version:    %perl_convert_version %{upstream_version}
> 

yes.

> If so, why does it need to be converted like that?

this is because 1.38 is considered 1.380.0 in perl land.
Comment 3 Shlomi Fish 2018-04-13 19:33:28 CEST
Assigning to QA for testing.

Status: NEW => ASSIGNED
Assignee: shlomif => qa-bugs

Comment 4 Herman Viaene 2018-04-15 13:38:31 CEST
MGA6-32 on Dell Latitude D600 MATE
No installation issues.
Test before update: I have my accounts in gnucash in Euro, added another dummy in AUD, and run the test as described above. I get this error. 
Closed gnucash, installed the update and do the test again. I get the same error.
I noticed that on this test laptop I have gnucash 2.6.19( seen as stable from MCC), while on my MGA6-64 desktop I have 2.6.18, and the newer version not seen in MCC.
This desktop is fully updated (not test repo's enabled).

CC: (none) => herman.viaene

Comment 5 Edward d'Auvergne 2018-04-18 10:23:41 CEST
Can the new 1.470 package also be put into "Core Updates Testing"?  I can only see it in "Core 32bit Updates Testing", hence I assume that is why I cannot test it.  Cheers!
Comment 6 Edward d'Auvergne 2018-04-18 10:56:14 CEST
Sorry, the 32bit package works.  I followed the instructions at:

    https://wiki.gnucash.org/wiki/FAQ#Q:_Why_doesn.27t_online_quoting_work.3F

The quote fetching works after obtaining a free key according to the above, and then running:

"""
$ export ALPHAVANTAGE_API_KEY=XXXXXXXXXXXXXX
$ gnc-fq-dump alphavantage CSCO
Finance::Quote fields Gnucash uses:
    symbol: CSCO                 <=== required
      date: 04/17/2018           <=== recommended
  currency: USD                  <=== required
      last: 44.5900              <=\       
       nav:                      <=== one of these
     price:                      <=/        
  timezone:                      <=== optional
"""

Now I need to work out why it doesn't work within Gnucash 2.6.18.  The instructions to go to "Tools->Security Editor" and changing the "Quote Source Information" to "unknown: alphavantage" does nothing.  It seems like this setting cannot be saved (editing the currency a second time shows that the setting is not saved).
Comment 7 Edward d'Auvergne 2018-04-18 11:23:03 CEST
More testing shows that the following command works when gnucash is not running and allows (at least some) of the quotes to be updated:

$ gnucash --add-price-quotes /path/to/file.gnucash

After this, I also updated to gnucash 2.6.19 as found in "Core Updates Testing".  This made no difference to the "gnucash --add-price-quotes" fetching or the inability to change the currency source to alphavantage in the Security Editor.
Comment 8 Edward d'Auvergne 2018-04-23 10:04:04 CEST
I've built the gnucash 2.6.21a tag from the git repository and the behaviour is the same as 2.6.19.  Fetching quotes via the GUI barely ever works, and the "--add-price-quotes" command line option only updates some of the quotes some of the time (though quite rarely).

I also built the "maint" branch which includes fixes on top of the 3.0 release (I couldn't build the 3.0 tag as it refused to find the GMOCK_ROOT installation path).  However "--add-price-quotes" results in a backtrace.  Fetching quotes from the GUI also has the same problem, 

I need to investigate further, but my guess is that the "Get Quotes" function is looping over the currencies, and terminating the loop if a quote fetching error occurs.  I have the feeling that certain currencies at the end of a long list can never be reached.  I might have to shift the conversation upstream.
Comment 9 Lewis Smith 2018-04-23 21:06:52 CEST
In the light of previous comments, I am feedbacking this until the situation is clearer. It would be nice to see the bug package named somewhere; one can deduce:
 perl-Finance-Quote-1.470.0-3.mga6
Also, it looks as as if Gnucash itself might need to be part of this update. If so, should it not be explicitly included & cited?

Keywords: (none) => feedback

Comment 10 Edward d'Auvergne 2018-04-26 09:22:18 CEST
From my testing, the new perl-Finance-Quote-1.470.0-3.mga6 package works reasonably well.  The Alphavantage quotes using their "key" works most of the time (as advertised, a heavy load on their server sometimes results in no quote being retrieved).  I used a Gnucash tool for the testing:

$ gnc-fq-dump currency USD XAU
1 USD = 0.00075451 XAU
$ gnc-fq-dump currency USD XAU
1 USD = 0.00075451 XAU
$ gnc-fq-dump currency USD XAU
Illegal division by zero at /usr/lib/perl5/vendor_perl/5.22.3/Finance/Quote.pm line 289.
$ gnc-fq-dump currency USD CNY
1 USD = <unknown> CNY
$ gnc-fq-dump currency USD CNY
1 USD = <unknown> CNY
$ gnc-fq-dump currency USD CNY
1 USD = <unknown> CNY
$ gnc-fq-dump currency CNY
$ gnc-fq-dump currency CNY EUR
1 CNY = 0.12991535 EUR
$ gnc-fq-dump currency CNY USD
1 CNY = 0.15822534 USD
$ gnc-fq-dump currency USD CNY
1 USD = 6.3201 CNY
$ 

Gnucash on the other hand is not doing so well.  My guess is that a single retrieval failure results in the rest of the current list not being updated.  I will report this separate issue upsteam.
Comment 11 Edward d'Auvergne 2018-04-26 09:32:22 CEST
Here is the GnuCash bug report:  https://bugzilla.gnome.org/show_bug.cgi?id=795564
Comment 12 Edward d'Auvergne 2018-05-15 12:35:05 CEST
I have also reported the issue upstream to Finance::Quote:  https://rt.cpan.org/Public/Bug/Display.html?id=125310
Comment 13 Dave Hodgins 2018-06-06 11:15:53 CEST
After following https://wiki.gnucash.org/wiki/FAQ#Q:_Why_doesn.27t_online_quoting_work.3F
and getting an api key for alphavantage, gnucash is working ok, at least for
Canadian/U.S conversion, with the update.

While there may be problems in gnucash working when the online service is not
responding, that's not a reason to hold this update.

I'm validating both this update and the gnucash update in bug 23036.

Blocks: (none) => 23036
Keywords: feedback => advisory, validated_update
Whiteboard: (none) => MGA6-64-OK
CC: (none) => davidwhodgins, sysadmin-bugs

Dave Hodgins 2018-06-06 11:16:11 CEST

Blocks: 23036 => (none)

Comment 14 Mageia Robot 2018-06-06 20:16:32 CEST
An update for this issue has been pushed to the Mageia Updates repository.

https://advisories.mageia.org/MGAA-2018-0109.html

Resolution: (none) => FIXED
Status: ASSIGNED => RESOLVED


Note You need to log in before you can comment on or make changes to this bug.