Bug 24243 - mythtv-setup: unable to automagically set xmltvid field
Summary: mythtv-setup: unable to automagically set xmltvid field
Status: RESOLVED WORKSFORME
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: Cauldron
Hardware: All Linux
Priority: Normal major
Target Milestone: ---
Assignee: Mageia Bug Squad
QA Contact:
URL: https://code.mythtv.org/trac/ticket/1...
Whiteboard:
Keywords: IN_ERRATA7, UPSTREAM
Depends on:
Blocks:
 
Reported: 2019-01-26 20:54 CET by Bit Twister
Modified: 2020-02-16 20:31 CET (History)
3 users (show)

See Also:
Source RPM: mythtv-30.0-20190121.1.mga7.tainted.src.rpm
CVE:
Status comment:


Attachments

Description Bit Twister 2019-01-26 20:54:56 CET
Description of problem:  mga7 beta1

mythtv-setup: unable to automagically set xmltvid field

On release 28 I would go through scan for channels to pull in channels
from my tuner, run a sql script to change channel number n_n to n.n, then
run  mythfilldatabase --dd-grab-all. At the end of that the xmltvid field
would have the SchedulesDirect channel id in xmltvid.

On release 30, mythfilldatabase would complain no grabber set, because of
the empty xmltvid field/column.

As a result there are no selections in the program guide.

Version-Release number of selected component (if applicable):


How reproducible: Always


Steps to Reproduce:
1. clean mythtv install
2. mythtv-setup
3. Channel Editor
4. delete all
5. scan and insert all
6. pick a channel and note empty xmltvid field

Using Schedules Direct for TV guide info/source.

Using Input Connections Fetch channels from listing source did not appear
to do anything other than complete.

Using a SiliconDust HDHR-US Tech network tuner.
Channel numbers come in as n_n. I wrote some sql code to flip the desired
channels to n.n.

Workaround for xmltvid was to pull it from a release 28 install
and fill it in the release 30 database.

User doing an upgrade will not notice the problem.
Comment 1 David GEIGER 2019-01-26 21:15:02 CET
This should be reported upstream!

https://code.mythtv.org/trac/wiki/TicketHowTo

CC: (none) => geiger.david68210

Bit Twister 2019-01-26 22:47:25 CET

URL: (none) => https://code.mythtv.org/trac/ticket/13388

Marja Van Waes 2019-01-31 18:53:44 CET

Keywords: (none) => UPSTREAM
CC: (none) => marja11
See Also: (none) => https://code.mythtv.org/trac/ticket/13388

Comment 2 Marja Van Waes 2019-02-05 15:01:00 CET
No feedback upstream, yet.

@ Bit Twister

Can you keep an eye on what happens upstream? This report needs to be assigned to a packager when upstream has a patch or a fixed version.
Comment 3 Bit Twister 2019-02-05 15:45:05 CET
(In reply to Marja Van Waes from comment #2)
> No feedback upstream, yet.
> 
> @ Bit Twister
> 
> Can you keep an eye on what happens upstream? 

I could, but it was easier to create a daily cron job/script to do it
for me.  :)
Comment 4 Bit Twister 2019-02-23 13:57:47 CET
Glad I wrote a script to scrape the bug report. Nothing on the screen showed
this embedded link rel="last" href="/trac/ticket/13411" title="Ticket #13411"
which gets me this surprise
    https://code.mythtv.org/trac/ticket/13411
Comment 5 Bit Twister 2019-02-23 23:37:17 CET
Received a message back from Schedules Direct.

I'm aware of it... a significant number of our members do use the MythTV (as well as I think half our board of directories).

As you mentioned,  MythTV is removing the native grabber and will only accept XMLTV formatted grabber.  There are three grabbers in the XMLTV project that obtain data from Schedules Direct.

tv_grab_na_dd  -- fetches the data direct feed.
tv_grab_zz_sdjson - fetches the sd-json feed (has additional fields and additional days)
tv_grab_zz_sdjson_sqlite  --  similar to the above... uses sqlite internally.

So, you just need to migrate to one of the other XMLTV grabbers and feed that into MythTV.  Many folks made the switch years ago... everyone else is just being pushed that way now.

Robert 
-----------------------------------------------------
https://www.mythtv.org/wiki/XMLTV suggests users need to install a XMLTV package.

Mageia does not seem to have one. :(

If packagers/devs have a deb to rpm converter you might look here
deb http://ftp.us.debian.org/debian sid main 

Users going to tv_grab_zz_sdjson will have to change XMLTV ids and need to
read https://www.mythtv.org/wiki/XMLTV
Comment 6 Bit Twister 2019-02-23 23:46:26 CET
(In reply to Bit Twister from comment #5)

> 
> Mageia does not seem to have one. :(


Ooops, I stand corrected, release 6 and 7 (cauldron) have the XMLTV package.
Comment 7 Marja Van Waes 2019-02-24 11:22:53 CET
(In reply to Bit Twister from comment #5)
> Received a message back from Schedules Direct.
> 
> I'm aware of it... a significant number of our members do use the MythTV (as
> well as I think half our board of directories).
> 
> As you mentioned,  MythTV is removing the native grabber and will only
> accept XMLTV formatted grabber.  There are three grabbers in the XMLTV
> project that obtain data from Schedules Direct.
> 
> tv_grab_na_dd  -- fetches the data direct feed.
> tv_grab_zz_sdjson - fetches the sd-json feed (has additional fields and
> additional days)
> tv_grab_zz_sdjson_sqlite  --  similar to the above... uses sqlite internally.
> 
> So, you just need to migrate to one of the other XMLTV grabbers and feed
> that into MythTV.  Many folks made the switch years ago... everyone else is
> just being pushed that way now.
> 
> Robert 
> -----------------------------------------------------
(In reply to Bit Twister from comment #6)
> (In reply to Bit Twister from comment #5)
> 
>  release 6 and 7 (cauldron) have the XMLTV package.

Does simply installing xmltv solve the problem, or does our xmltv package need an adjustment (on top of making it depend on xmltv)?
Comment 8 Bit Twister 2019-02-24 13:18:50 CET
(In reply to Marja Van Waes from comment #7)

> (In reply to Bit Twister from comment #6)
> > (In reply to Bit Twister from comment #5)
> > 
> >  release 6 and 7 (cauldron) have the XMLTV package.
> 
> Does simply installing xmltv solve the problem, or does our xmltv package
> need an adjustment (on top of making it depend on xmltv)?

Lucky you, I just now completed my xmltv install/changes on my test bed node.

Me thinks you might have meant mythtv backend package wanting to also
pull in xmltv, xmltv-grabbers-zz_sdjson, and myth web pull in xmltv-gui.

I installed xmltv, xmltv-grabbers-zz_sdjson, and not needed xmltv-gui packages.
mythtv web users may want the gui.

Myth database was a mga7 clean network install with restored database
from backup, after my manual workaround scripts, everything was normal.
That setup is now running on all three nodes.

Installed xmltv packages, configured xmltv-grabbers-zz_sdjson to use 
schedulesdirect, verified graber could pull down guide info.

Changed grabber from schedulesdirect to xmltv-grabbers-zz_sdjson in
mythtv-setup, deleted channel scan data, re-scaned for channels from
my network tuner, and all channel's xmltvid were null.

Started mythbackend, ran mythfilldatabase --dd-grab-all and the 
channel xmltvid field had data for desired viewing channels.

That is the good news and shows that are recorded are still there for viewing.
Bad news, all shows picked for upcomming recording are gone.

I will have to go through the guide and pick them again for recording.
Downside of that is I will probably get all the watched shows recorded again.

Users need to read https://www.mythtv.org/wiki/XMLTV for background
information warnings and whatnot, then read 
  https://blogging.dragon.org.uk/schedules-direct-uk/
which made my transition painless.
Comment 9 Bit Twister 2019-02-24 18:39:57 CET
(In reply to Bit Twister from comment #8)
> (In reply to Marja Van Waes from comment #7)

I just now completed my "Production" node conversion to XMLTV usage.
Looks good, guide has selections, upcoming recording are still there.


> Bad news, all shows picked for upcoming recording are gone.
> 
> I will have to go through the guide and pick them again for recording.

Not true. upcoming recordings are still there. I had forgotten I had deleted
everything in the recording screen in a previous test shot on test bed node.

Looks like an errata entry about SchedulesDirect users need to setup
xmltv to download guide if not not already using it.

Users need to read https://www.mythtv.org/wiki/XMLTV for background
information warnings and whatnot, then read 
  https://blogging.dragon.org.uk/schedules-direct-uk/
which made my transition painless.

Note: Everything is in mythtv's home directory.
To get home directory location run:
   getent passwd "mythtv" | cut -d: -f6


When I ran tv_grab_zz_sdjson --configure. I set the 
Cache File: directory as /var/lib/mythtv/.xmltv

My video SOURCENAME is Antenna, So I created the link
Antenna.xmltv -> /var/lib/mythtv/.xmltv/tv_grab_zz_sdjson.conf
in /var/lib/mythtv

No idea if you want to reference Comment 8 and Comment 9 in errata.
Comment 10 Lewis Smith 2020-02-02 16:23:47 CET
Bit Twister
Mageia 7 is long out. What is the state of play on this bug? Can we close it?
> SchedulesDirect users need to setup
> xmltv to download guide if not not already using it.
> Users need to read
>  https://www.mythtv.org/wiki/XMLTV for background
> information warnings and whatnot, then read 
>  https://blogging.dragon.org.uk/schedules-direct-uk/
You mentioned ERRATA; did it get there? But this could perhaps be in:
 /usr/share/doc/mythtv-doc/README
 /usr/share/doc/mythtv-doc/UPGRADING
if not already done.

The application is heavy to install just to look at those. Could you please check whether these files are adequate as they are? If not, I imagine your comments could be added therein.
--------------------------------
From your comment 8:
> mythtv-backend package wanting to also
> pull in xmltv, xmltv-grabbers-zz_sdjson
 Neither currently. Nothing requires 'xmltv' nor 'xmltv-grabbers-zz_sdjson'.
> myth[tv]-mythweb pull in xmltv-gui
 Not currently. Nor is xmltv-gui required by anything.

If you are confident of solid 'requires', you could post your ideas as a separate 'enhancement' bug.

CC: (none) => lewyssmith

Comment 11 Bit Twister 2020-02-02 18:43:43 CET
(In reply to Lewis Smith from comment #10)
> Bit Twister
> Mageia 7 is long out. What is the state of play on this bug? Can we close it?
> > SchedulesDirect users need to setup
> > xmltv to download guide if not not already using it.
> > Users need to read
> >  https://www.mythtv.org/wiki/XMLTV for background
> > information warnings and whatnot, then read 
> >  https://blogging.dragon.org.uk/schedules-direct-uk/
> You mentioned ERRATA; did it get there? But this could perhaps be in:
>  /usr/share/doc/mythtv-doc/README
>  /usr/share/doc/mythtv-doc/UPGRADING
> if not already done.
> 
> The application is heavy to install just to look at those.

No mention about schedule direct removal in either document.

> Could you please
> check whether these files are adequate as they are? If not, I imagine your
> comments could be added therein.

Mageia 7 release/errata have no mythtv entries.

> --------------------------------
> From your comment 8:
> > mythtv-backend package wanting to also
> > pull in xmltv, xmltv-grabbers-zz_sdjson
>  Neither currently. Nothing requires 'xmltv' nor 'xmltv-grabbers-zz_sdjson'.
> > myth[tv]-mythweb pull in xmltv-gui
>  Not currently. Nor is xmltv-gui required by anything.
> 
> If you are confident of solid 'requires', you could post your ideas as a
> separate 'enhancement' bug.

No, solid 'requires'. xmltv are based on country. unless using Schedules Direct
so it is user's choice on which one is needed.

The errata maintenance/process seems to have a problem since the mga6 mythtv
errata entry was dropped when going to mga7 about bug 19127.

Guessing simple fix here would be to create a /usr/share/doc/mythtv
 README.mageia file about this, reference Comment 8, and the other
bug and include it in the mythtv common rpm.

PS: You may want to consider changing Assignee: on bug 19127.

If both bugs were placed in mga7 errata. I see no reason to keep this
bug open.
Lewis Smith 2020-02-03 12:03:41 CET

See Also: (none) => https://bugs.mageia.org/show_bug.cgi?id=19127

Comment 12 Lewis Smith 2020-02-03 12:05:33 CET
It looks as if this & bug 19127 could be fixed by either adapting 
 /usr/share/doc/mythtv-doc/README
 /usr/share/doc/mythtv-doc/UPGRADING
with the necessary instructions, if this is permitted;
or create as suggested a
 /usr/share/doc/mythtv/README.mageia file about this
reference Comment 4 and comment 8.

The registered maintainer for mythtv is no longer active, so assigning this globally. DavidG is already involved.
Doubtful that we enhance ERRATA post release, but just in case this is possible, CC'ing Atelier.
Comment 13 Lewis Smith 2020-02-08 10:12:54 CET
@Bit Twister :
It seems we *can* still add to M7 ERRATA.

For bug 19127, from https://bugs.mageia.org/show_bug.cgi?id=19127#c12
"mga7 users need just the above comment_#11 workaround for mythtv-setup and mythbackend":
 https://bugs.mageia.org/show_bug.cgi?id=19127#c11
That is easy to include. Should it be in the package installation?

For *this* bug, I cannot see exactly what text would be required from your various comments. Can you please state the text you think is necessary for the Errata (or an extra README)? Somewhere among comment 8 & comment 9.
Again, can the solution be packaged?

I will do the M7 ERRATA once I can see what to put.
Comment 14 Bit Twister 2020-02-08 12:44:48 CET
(In reply to Lewis Smith from comment #13)
> @Bit Twister :

> Again, can the solution be packaged?

Only if Mageia wants to create a meta package just for Schedules Direct users.
I would not recommend doing so.
 
> I will do the M7 ERRATA once I can see what to put.

My suggestion:

MythTV no longer supports the direct grab Schedules Direct software.
Schecules Direct users need to install xmltv grabber packages:

xmltv, xmltv-grabbers-zz_sdjson, and not needed xmltv-gui packages.
mythtv web users may want the gui package.


Users need to read https://www.mythtv.org/wiki/XMLTV for background
information warnings and whatnot, then read 
  https://blogging.dragon.org.uk/schedules-direct-uk/
which made my transition painless.

Note: Everything is in mythtv's home directory.
To get home directory location run:
   getent passwd "mythtv" | cut -d: -f6


When running "tv_grab_zz_sdjson --configure"  set the 
Cache File: directory as /var/lib/mythtv/.xmltv

My video SOURCENAME is Antenna, So I created the link
Antenna.xmltv -> /var/lib/mythtv/.xmltv/tv_grab_zz_sdjson.conf
in /var/lib/mythtv

For more information, see https://bugs.mageia.org/show_bug.cgi?id=24243
Comment 15 Lewis Smith 2020-02-09 22:02:06 CET
Thank you for the suggestion above. Some clarifications please:
When you say "and not needed xmltv-gui packages" do you mean they are optional?
"mythtv web users may want the gui package" is it too a sort of recommendation?

What about also (from bug 19127 comments 11-12):
> mga7 users need just the following workaround for mythtv-setup and mythbackend.:
> echo "MYTHTV_HOME=/var/lib/mythtv" >> /etc/sysconfig/mythbackend
> echo "MYTHCONFDIR=/var/lib/mythtv" >> /etc/sysconfig/mythbackend
Does this still apply?

TIA
Comment 16 Bit Twister 2020-02-09 22:39:21 CET
(In reply to Lewis Smith from comment #15)
> Thank you for the suggestion above. Some clarifications please:
> When you say "and not needed xmltv-gui packages" do you mean they are
> optional?

Yes. Not needed if using the command line or creating/using a script.

> "mythtv web users may want the gui package" is it too a sort of
> recommendation?

Yes, Not all mythtv users are installing/using web interface.

> What about also (from bug 19127 comments 11-12):
> > mga7 users need just the following workaround for mythtv-setup and mythbackend.:
> > echo "MYTHTV_HOME=/var/lib/mythtv" >> /etc/sysconfig/mythbackend
> > echo "MYTHCONFDIR=/var/lib/mythtv" >> /etc/sysconfig/mythbackend
> Does this still apply?

Going to guess no. I checked and mine has the two lines, and I can
find no code creating the lines in my install/change scripts.
Comment 17 Lewis Smith 2020-02-15 20:39:17 CET
See https://wiki.mageia.org/en/Mageia_7_Errata#MythTV . Is it OK?
From c11:
> If both bugs were placed in mga7 errata. I see no reason to keep this
> bug open.
It seems from the last item in the previous comment that bug 19127 is no longer current; so not possible to mention in Errata.
In any case, can we now close this bug at least? Ideally, both.
Comment 18 Bit Twister 2020-02-15 22:42:57 CET
(In reply to Lewis Smith from comment #17)
> See https://wiki.mageia.org/en/Mageia_7_Errata#MythTV . Is it OK?
> From c11:
> > If both bugs were placed in mga7 errata. I see no reason to keep this
> > bug open.
> It seems from the last item in the previous comment that bug 19127 is no
> longer current; so not possible to mention in Errata.


Hmmm, pretty sure item one in bug 19127 
1. passwords in /usr/share/mythtv/initialdb/mc.sql need to be 8 characters or longer.

is the reason the bug is not closed one way or the other.

> In any case, can we now close this bug at least? 

Ok, by me.


> Ideally, both.

If errata warns user about needing to change password in scripts I'll leave
it up to Mageia dev/packager to decide.

I modified password in scripts and the install works for me, so I no longer
care.
Comment 19 Lewis Smith 2020-02-16 20:31:50 CET
So I am closing this bug because what *it* points up is in ERRATA7. It looks as if to tidy this up, we need to transfer attention to Bug 19127.

> pretty sure item one in https://bugs.mageia.org/show_bug.cgi?id=19127#c0
> 1. passwords in /usr/share/mythtv/initialdb/mc.sql need to be 8 characters
> or longer.
> is the reason the bug is not closed one way or the other
I have asked on *that* bug what you think needs to still be in ERRATA7. The mere question of passwords looks a MariaDB/MySQL issue.

Keywords: (none) => IN_ERRATA7
Resolution: (none) => WORKSFORME
Status: NEW => RESOLVED


Note You need to log in before you can comment on or make changes to this bug.