Bug 338 - We need a way to have a package built for several targets
Summary: We need a way to have a package built for several targets
Status: ASSIGNED
Alias: None
Product: Infrastructure
Classification: Unclassified
Component: BuildSystem (show other bugs)
Version: unspecified
Hardware: i586 Linux
Priority: Normal normal
Target Milestone: Mageia 1
Assignee: Sysadmin Team
QA Contact:
URL:
Whiteboard:
Keywords:
: 1173 (view as bug list)
Depends on:
Blocks: 858 577 1173 1192 1204 1331 1377
  Show dependency treegraph
 
Reported: 2011-03-11 11:58 CET by Pascal Terjan
Modified: 2014-03-24 10:52 CET (History)
13 users (show)

See Also:
Source RPM:
CVE:
Status comment:


Attachments

Description Pascal Terjan 2011-03-11 11:58:59 CET
Some packages need to be built for both core and tainted

Reproducible: 

Steps to Reproduce:
Pascal Terjan 2011-03-11 11:59:24 CET

Priority: Normal => High
Severity: normal => critical

Comment 1 Pascal Terjan 2011-03-17 18:04:01 CET
I had started some experiment and so far this is one of the best (but ugly) way I found to get the info...

We would then need to have the submit process define querytarget and get the info from stderr (we can also change the description when this macro is defined, or whatever way to export it, %error was the only one I found to display)

We would also need to change iurt to define mga_media

# Non package specific stuff
%define targetmedia(%1) %{?querytarget:%{error:TARGETMEDIA %1}}
%{?!mga_media:%define mga_media core}

%targetmedia core,tainted

Summary: Foo
Name: foo
Version: 1
Release: 1
License: WTFPL
Group: Development/Other
Url: http://127.0.0.1/


%if %mga_media == tainted
#...
%endif

%description
Built for %mga_media

%build

%files
Ahmad Samir 2011-04-12 08:39:18 CEST

Blocks: (none) => 577

Thierry Vignaud 2011-04-14 20:58:52 CEST

CC: (none) => thierry.vignaud

Romain d'Alverny 2011-04-14 22:31:49 CEST

CC: (none) => rdalverny

Tux99 2011-04-16 13:23:11 CEST

CC: (none) => tux99

Nicolas Vigier 2011-04-17 17:37:04 CEST

Blocks: (none) => 858

Anssi Hannula 2011-04-27 03:54:44 CEST

CC: (none) => anssi.hannula

Comment 2 Anssi Hannula 2011-04-27 03:56:42 CEST
Upgrading priority to a release blocker, we need this one to resolve the codecs situation before Mageia 1.

Priority: High => release_blocker

Romain d'Alverny 2011-04-27 10:08:26 CEST

Target Milestone: --- => Mageia 1

Comment 3 Anssi Hannula 2011-04-27 16:11:20 CEST
Pascal's suggestion seems reasonable to me. The non-package-specific stuff could maybe be put in common rpm macros.
Comment 4 Pascal Terjan 2011-04-28 03:58:31 CEST
Another solution could be to use some virtual packages target-mageia-foo for each media, and then if no target is buildrequired, build for core, else build for all required targets

The targets could be provided by rpm-mageia-setup-build 

I still don't really like it but it is simpler
Comment 5 Michael Scherer 2011-04-30 22:55:36 CEST
I would prefer to have the target listed outside of the spec, as this is rather BS specific, but this could be changed later.

CC: (none) => misc

Comment 6 Pascal Terjan 2011-04-30 23:12:50 CEST
Well we need a way in the spec to know the target to decide of the buildrequires, configure options, file list etc. So this will be specific anyway.
Comment 7 Pascal Terjan 2011-05-01 10:29:00 CEST
Thinking about your comment, default could easily be "where it is currently existing" instead of defaulting to core.

Another thing I just thought was, should we duplicated the src.rpm ?
Comment 8 Pascal Terjan 2011-05-01 21:38:59 CEST
We can also try to extend the code to look for the medias where the package exists if no sections are given and have no information in the package itself about requested sections
Comment 9 Michael Scherer 2011-05-02 17:31:24 CEST
Duplicate the src.rpm on mirror ?

If we don't duplicate, tools that would seek src.rpm from binary would likely break, or requires some specific hook.

On the other hand, copying the src.rpm would waste space ( can be done with hardlink ), and bring a src.rpm that would not exactly correspond to the binary rpm ( as it requires a switch ). 

But I would vote "duplicate", as it would cause trouble on tools we already use ( youri ).
Comment 10 Pascal Terjan 2011-05-04 11:36:20 CEST
Yesterday in the train I patched (not tested or committed yet) ulri and started emi, to support multiple targets for the same prefix.

However I noticed a big problem... we will end up with several packages with same name/version/release/... and no way to update from one to the other or to prefer one.

I could not think of a nice solution...
One ugly hack could be to sort target media (core/nonfree/tainted) and override mkrel in all targets of the package except the first one to append something (increasing) to release but I find this really ugly...
Comment 11 Tux99 2011-05-04 11:41:34 CEST
I though the plan (based on discussions on the ML about this many months ago) was to use "mgt" for tainted instead of "mga", which would make urpmi prefer tainted when enabled just like "plf" is preferred to "mdv".
Comment 12 Pascal Terjan 2011-05-04 11:43:47 CEST
I never saw that discussion, that can be an option but currently packages in tainted are mga AFAIK
Comment 14 Thierry Vignaud 2011-05-04 12:12:04 CEST
They can still be rebuild with the new mkrel
Comment 15 Nicolas Vigier 2011-05-04 12:28:35 CEST
When there is more than one version of a package available, I think it would be useful to know which version of the package is installed. So using release tag could be a solution.

Maybe mkrel can be updated to use %mga_media ?

CC: (none) => boklm

Comment 16 Pascal Terjan 2011-05-04 12:31:33 CEST
Or we can not add %mga_media and do %if %distsuffix == ...
Comment 17 Michael Scherer 2011-05-05 23:48:40 CEST
Distsuffix seems the best solution.

We can even have rpm-mageia-setup compiled as a dual life rpm ( with %distsuffix = mga in core, and %distuffix = $another_distsuffix in tainted ), so when we setup the iurt chroot, the tainted version will be installed and change the distsuffix without any change to iurt.

The contingency plan, if the multiple submit is not ready, is just to duplicate the srpm, I guess. Suboptimal, but it would work.
Ahmad Samir 2011-05-06 21:14:19 CEST

Blocks: (none) => 1173

Comment 18 Frank Griffin 2011-05-06 21:50:27 CEST
*** Bug 1173 has been marked as a duplicate of this bug. ***

CC: (none) => ftg

Comment 19 Christiaan Welvaart 2011-05-07 14:49:12 CEST
A 'tainted' rpm-mageia-setup sounds interesting but what would happen if a packager has the 'tainted' medium enabled on a build machine - all packages will be built with .mgt release suffix. Probably not a real problem, but not something people will expect.

Where a package comes from is something urpmi (or in my case apt) already knows, the only issue here is that the 'tainted' package must be different and preferred over the 'core' version. And the release tag is the only simple solution for that.

Having distsuffix checks in the specfile doesn't sound good - those suffixes are quite arbitrary. How about the build system simply passes the relevant info, for example:

 --with patents  # for tainted medium
 --with nonfree  # for nonfree medium
 --with patents --with nonfree  # for a new 'restricted' medium

We could leave the release suffix up to the packages for now (like with PLF) and implement medium-specific release suffixes after mageia 1 release - so initially, tainted-only packages have release suffix .mga, while packages like mplayer use .mgt .

To support all combinations we would have to use .mga/.mgb/.mgc/.mgd or .mga/.mgn/.mgt/.mgx  for core/nonfree/tainted/restricted .

CC: (none) => cjw

Comment 20 Ahmad Samir 2011-05-07 17:58:36 CEST
(In reply to comment #19)
> A 'tainted' rpm-mageia-setup sounds interesting but what would happen if a
> packager has the 'tainted' medium enabled on a build machine - all packages
> will be built with .mgt release suffix. Probably not a real problem, but not
> something people will expect.

Packagers could have the rpm-mageia-setup from tainted in skip.list... packages built locally on one's machines aren't uploaded to the repos anyway; packages in the repos have to go through the BS, if the BS works, it doesn't matter much..

> 
> Where a package comes from is something urpmi (or in my case apt) already
> knows, the only issue here is that the 'tainted' package must be different and
> preferred over the 'core' version. And the release tag is the only simple
> solution for that.
> 
> Having distsuffix checks in the specfile doesn't sound good - those suffixes
> are quite arbitrary. How about the build system simply passes the relevant
> info, for example:
> 
>  --with patents  # for tainted medium
>  --with nonfree  # for nonfree medium
>  --with patents --with nonfree  # for a new 'restricted' medium

[....]

> 
> We could leave the release suffix up to the packages for now (like with PLF)
> and implement medium-specific release suffixes after mageia 1 release - so
> initially, tainted-only packages have release suffix .mga, while packages like
> mplayer use .mgt .
> 

All packages in tainted will have the same distsuffix, IIUC; it doesn't matter if it's a tainted only package or a package that exists in core and tainted at the same time.

> To support all combinations we would have to use .mga/.mgb/.mgc/.mgd or
> .mga/.mgn/.mgt/.mgx  for core/nonfree/tainted/restricted .

Having two distsuffix'es is already an ugly hack but it's one of the few available options (the other, as suggested by blino, was to change the name, i.e. mplayer in core and mplayer-tainted in tainted.... still a bit on an ugly hack; just one hack should be selected); having a distsuffix for each repo, isn't useful IMHO, nothing is gained from introducing more complexity.
José Jorge 2011-05-07 19:16:05 CEST

Blocks: (none) => 1192

Ahmad Samir 2011-05-09 22:52:10 CEST

Blocks: (none) => 1204

Ahmad Samir 2011-05-11 02:26:22 CEST

Blocks: (none) => 985

Sébastien GUERIN 2011-05-16 10:35:04 CEST

CC: (none) => sebastien.guerin.news

Comment 21 Anssi Hannula 2011-05-17 20:05:57 CEST
For the record, here's blino's take (which I may or may not agree with):

< blino> changing distsuffix is ugly...
< blino> packages come from the same vendor, distsuffix should remain the same
< blino> we could have some youri plugin to automatically move packages with "Provides: tainted-something" in the tainted (or non-free) section
< blino> like we do with debug packages
< blino> it would allow to easily have free + non-free/tainted stuff from the same src.rpm
Comment 22 Pascal Terjan 2011-05-17 20:09:42 CEST
The problem is not about repository but about users.
We _need_ to have something different in either name or version or release, else we install a random package and don't even know which one we get.
Changing version is the worse. Changing release seems the best.
Comment 23 Anssi Hannula 2011-05-17 20:12:36 CEST
Blino's idea was to have them with different name (-tainted in name or so).

Obviously the big downside is that without additional integration in tools it is rather hard to switch to tainted versions of all packages.

In the long run it'd seem reasonable if integrated properly, but indeed I don't think it is possible in the current timeframe.
Comment 24 Pascal Terjan 2011-05-17 20:16:22 CEST
And I actually does not think there is a reason to force distsuffix == vendor...

Having for example mgat meaning "Mageia Tainted packages" does not seem wrong
Comment 25 Anssi Hannula 2011-05-17 20:18:51 CEST
Yep.
Comment 26 Christiaan Welvaart 2011-05-18 00:47:52 CEST
For the record: next to name, version and release, there's also epoch, and increasing epoch by 1 for tainted packages might work and would not show up in file names. I think that would be a terrible idea, however. Tweaking the release makes more sense.

IMHO an extension should come after the dist version, so:
  <dist name><dist version><dist section>
for example full release number suffix ".mga1.t" for tainted.

But ".mgat1" would also work I guess.
Ahmad Samir 2011-05-19 23:59:26 CEST

Blocks: (none) => 1331

Barry Jackson 2011-05-20 00:01:29 CEST

Blocks: 1331 => (none)

Barry Jackson 2011-05-20 12:23:04 CEST

Blocks: (none) => 1331

John Balcaen 2011-05-22 10:16:12 CEST

Blocks: (none) => 1377

Comment 27 José Jorge 2011-05-28 14:36:18 CEST
Now that FAAD is in tainted, will VLC (and other players) be rebuilt against it?
Isn't that needed to have a 1 release that plays AAC?

CC: (none) => lists.jjorge

Marcello Anni 2011-05-28 15:41:14 CEST

CC: (none) => marcello.anni

Comment 28 Ahmad Samir 2011-05-28 22:15:34 CEST
(In reply to comment #27)
> Now that FAAD is in tainted, will VLC (and other players) be rebuilt against
> it?
> Isn't that needed to have a 1 release that plays AAC?

Sure, but first this report "We need a way to have a package built for several targets" needs to be resolved before that happens.
Comment 29 Pascal Terjan 2011-05-29 17:09:12 CEST
It sort of works but I think we should improve the mkrel macro to not set it if it distro_section is core, and have it defined to core if undefined.

Currently I have set iurt to define distro_section except when section is core, and have to test existence of the macro in the spec file before testing its value else it would fail to build for core.
Comment 30 Pascal Terjan 2011-05-29 17:14:43 CEST
Such change should be enough in many packages as a first step:

[pterjan@pterjan-glaptop ffmpeg]$ svn diff
Index: SPECS/ffmpeg.spec
===================================================================
--- SPECS/ffmpeg.spec	(revision 100671)
+++ SPECS/ffmpeg.spec	(working copy)
@@ -24,6 +24,10 @@
 %if %build_plf
 %define distsuffix plf
 %endif
+%if "%distro_section" == "tainted"
+%global build_plf 1
+%endif
+
 %define build_faac	0
 %{?_with_faac: %{expand: %%global build_faac 1}}
 %{?_without_faac: %{expand: %%global build_faac 0}}
Comment 31 Christiaan Welvaart 2011-05-29 17:55:30 CEST
Leaving out the distro section suffix for core in mkrel would be useful, but what would the necessary rpm macro magic look like?
Barry Jackson 2011-07-01 16:12:00 CEST

Blocks: (none) => 1987

Ahmad Samir 2011-07-01 22:08:31 CEST

Blocks: 1987 => (none)

Comment 32 Philippe Didier 2011-09-19 12:56:43 CEST
It seems this old bug report may be closed as we can see packages being built for core & tainted, 
Is it still valid ?


By the way 

Colin has found an elegant way to modify spec file allowing tainted build 
here's are extract from diffs



15 %define build_plf    0 	         %if "%distro_section" == "tainted"
16 	  	                         %global build_tainted 1
17 %bcond_with plf 	                 %else
18 %if %with plf 	                 %global build_tainted 0
   %define distsuffix   plf 	 
   %define build_plf    1 	 


45 %if %build_plf 	                 %if %{build_tainted}
46 BuildRrequires xxx-devel              BuildRrequires xxx-devel
47 %endif                                %endif


76 %if %with plf 	                 %if %{build_tainted}
77 This package is in PLF because        This package is in the tainted section
 it contains software that supports      because it contains software that   
                                         supports  
78 codecs that may be covered            codecs that may be covered
 by software patents. 	                 by software patents.
79 %endif                                %endif

CC: (none) => philippedidier

Comment 33 Pascal Terjan 2011-09-19 12:59:16 CEST
There is still a part missing, you currently need to manually submit for each target
Comment 34 Philippe Didier 2011-09-19 13:38:22 CEST
Ok 
Sorry ! 
Nevertheless this workaround allows to have a real choice for the users : the solution found for the core and tainted suffixes makes this less critical and no more release blocker... which sounds bad when you scroll the bug list :(
Comment 35 Pascal Terjan 2011-09-19 14:01:00 CEST
Indeed, lowered severity

Priority: release_blocker => Normal
Severity: critical => normal

Colin Guthrie 2011-09-20 10:11:06 CEST

CC: (none) => mageia

Philippe Didier 2011-09-22 00:38:36 CEST

Blocks: 985 => (none)

Comment 36 Marja Van Waes 2011-12-31 19:19:57 CET
pinging. because nothing happened to this report since more than 3 months ago, and it still has the status NEW


@ sysadmins
Please set status to ASSIGNED if you think this bug was assigned correctly. If for work flow reasons you can't do that, then please put OK on the whiteboard instead.

CC: (none) => marja11

Nicolas Vigier 2012-01-01 16:50:52 CET

Status: NEW => ASSIGNED

Sébastien GUERIN 2012-01-01 23:25:59 CET

CC: sebastien.guerin.news => (none)

Götz Waschk 2012-10-01 09:32:58 CEST

CC: (none) => goetz.waschk

Nicolas Vigier 2014-03-24 10:49:00 CET

CC: boklm => (none)


Note You need to log in before you can comment on or make changes to this bug.