Bug 952 - mgarepo up doesn't handle conflicts well
Summary: mgarepo up doesn't handle conflicts well
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: 3
Hardware: i586 Linux
Priority: Normal normal
Target Milestone: ---
Assignee: QA Team
QA Contact:
URL:
Whiteboard: MGA2TOO has_procedure MGA2-32-ok MGA2...
Keywords: validated_update
: 6764 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-04-23 15:05 CEST by Anssi Hannula
Modified: 2014-05-08 18:07 CEST (History)
10 users (show)

See Also:
Source RPM: mgarepo-1.9.9
CVE:
Status comment:


Attachments
Poll pipe in noblocking mode (1.77 KB, patch)
2013-06-30 02:48 CEST, Dan Fandrich
Details | Diff
Stop doing blocking reads (1.24 KB, patch)
2013-06-30 03:01 CEST, Dan Fandrich
Details | Diff

Description Anssi Hannula 2011-04-23 15:05:19 CEST
$ mgarepo up
D    SOURCES/0001-Removed-duplicate-code-block-mis-merge-prolly.patch
Conflict discovered in 'SPECS/libreoffice.spec'.
Select: (p) postpone, (df) diff-full, (e) edit,
        (mc) mine-conflict, (tc) theirs-conflict,
        (s) show all options: D    SOURCES/kde4configure.patch
D    SOURCES/0001-helgrind-Related-rhbz-655686-get-order-of-shutdown-c.patch
D    SOURCES/185d60944ea767075d27247c3162b3bc-unowinreg.dll
D    SOURCES/0001-bubble-down-configure-test-findings-on-visibility.patch
D    SOURCES/0001-Resolves-rhbz-695509-crash-in-RefreshDocumentLB.patch
U    SOURCES/openoffice.org-2.4.0.ooo86080.unopkg.bodge.patch
A    SOURCES/libreoffice-bootstrap-kde.patch
[...] lots of files [...]
A    SOURCES/rhbz680766.fix-mdds-crash.patch
U    SOURCES/openoffice.org-3.0.0.ooo88341.sc.verticalboxes.patch
A    SOURCES/mdds.add-missing-link.patch
A    SOURCES/openoffice.org-3.1.0.ooo102061.sc.cellanchoring.patch
e
Vim: Varoitus: Tuloste ei mene terminaalille


As you can see, there are two (maybe related) issues:
1) mgarepo continues to update stuff even after prompting for conflict recovery action.
2) when selecting 'e' for edit, it tries to start the editor but there is a warning ~"Output not a terminal", and indeed the editor screen doesn't appear.
Ahmad Samir 2011-04-23 19:40:58 CEST

Assignee: bugsquad => boklm

Comment 1 Colin Guthrie 2011-09-29 15:27:51 CEST
For me I don't even see the question to resolve conflicts.


mgarepo just seems to hang indefinitely. I hit ctrl + C then svn up, resolve then rerun mgarepo (which syncs binaries) and all is well.

CC: (none) => mageia

Comment 2 Marja Van Waes 2011-12-31 19:33:25 CET
any news about this bug?

CC: (none) => marja11

Comment 3 Florian Hubold 2012-02-14 14:02:41 CET
(In reply to comment #1)
> For me I don't even see the question to resolve conflicts.
> 
> 
> mgarepo just seems to hang indefinitely. I hit ctrl + C then svn up, resolve
> then rerun mgarepo (which syncs binaries) and all is well.

Same behaviour for me with mgarepo-1.10.0-2.mga1.

CC: (none) => doktor5000

Comment 4 Marja Van Waes 2012-05-26 13:05:50 CEST
Hi,

This bug was filed against cauldron, but we do not have cauldron at the moment.

Please report whether this bug is still valid for Mageia 2.

Thanks :)

Cheers,
marja

Keywords: (none) => NEEDINFO

Comment 5 Marja Van Waes 2012-07-06 15:06:20 CEST
Please look at the bottom of this mail to see whether you're the assignee of this  bug, if you don't already know whether you are.


If you're the assignee:

We'd like to know for sure whether this bug was assigned correctly. Please change status to ASSIGNED if it is, or put OK on the whiteboard instead.

If you don't have a clue and don't see a way to find out, then please put NEEDHELP on the whiteboard.

Please assign back to Bug Squad or to the correct person to solve this bug if we were wrong to assign it to you, and explain why.

Thanks :)

**************************** 

@ the reporter and persons in the cc of this bug:

If you have any new information that wasn't given before (like this bug being valid for another version of Mageia, too, or it being solved) please tell us.

@ the reporter of this bug

If you didn't reply yet to a request for more information, please do so within two weeks from now.

Thanks all :-D
Comment 6 Marja Van Waes 2012-08-04 16:51:14 CEST
2x no reply, closing as old.

Please reopen when needed and tell for which version(s) of Mageia and mageiarepo this bug is still valid

If it is valid for both cauldron and Mageia 2, then please put MGA2TOO on the whiteboard and let version remain set to cauldron

Status: NEW => RESOLVED
Resolution: (none) => OLD

Comment 7 Nicolas Vigier 2012-08-04 17:48:10 CEST
Why do you need to close this bug as old ?
Comment 8 Marja Van Waes 2012-08-04 18:06:12 CEST
(In reply to comment #7)
> Why do you need to close this bug as old ?

I wasn't aware that this bug was for Mageia 1 too.

Reading the comments, the only thing I'm sure of is that the bug is valid for Mageia 1, for mgarepo-1.10.0-2.mga1

So reopening and setting version to 1

If you know it is still valid for mgarepo-1.10.2-1.mga2, so for cauldron and Mageia 2, then please change version back to cauldron and put MGA2TOO and MGA1TOO on the whiteboard.

Keywords: NEEDINFO => (none)
Status: RESOLVED => REOPENED
Version: Cauldron => 1
Resolution: OLD => (none)

Nicolas Vigier 2012-08-13 16:53:53 CEST

Version: 1 => Cauldron

Marja Van Waes 2012-08-13 18:25:29 CEST

Whiteboard: (none) => MGA1TOO, MGA2TOO

Comment 9 Manuel Hiebel 2012-10-20 22:04:07 CEST
boklm leave Mageia

Status: REOPENED => NEW
Assignee: boklm => bugsquad

Comment 10 Nicolas Lécureuil 2013-03-27 11:18:51 CET
this is indeed a bug to fix.

i will try to reproduce

CC: (none) => nicolas.lecureuil

Comment 11 Dan Fandrich 2013-06-30 02:04:11 CEST
The problem stems from the update() function in rpmutil.py calling svn.update with show=True. This causes the --non-interactive argument to NOT be included on the svn up command-line, so, presumably, when svn hangs it is actually waiting for manual intervention about the conflict. Presumably, there is buffering going on in execcmd() which prevents the output from being seen. I experimented with setting the fds in execcmd() to non-blocking mode, and that fixed this problem, but the read/write loop busy waited. This may be the way to solve it, though.

CC: (none) => dan

Comment 12 Dan Fandrich 2013-06-30 02:48:18 CEST
Created attachment 4178 [details]
Poll pipe in noblocking mode

The attached patch fixes the problem for me.
Comment 13 Dan Fandrich 2013-06-30 03:01:16 CEST
Created attachment 4179 [details]
Stop doing blocking reads

On reflection, the fds don't even need to be set nonblocking. This patch is even simpler.

Attachment 4178 is obsolete: 0 => 1

Comment 14 Dan Fandrich 2013-06-30 12:08:34 CEST
The originally-reported issue 2) where "e" fails to start an editor still isn't completely fixed with this patch, as vim still says "Vim: Warning: Output is not to a terminal" (which is true--the svn subprocess is run through a pipe, not a pty), but vim at least acts semi-sanely and mostly works (the cursor doesn't show for me) but at least lets you exit to select another merge option.
Comment 15 Dan Fandrich 2013-07-02 22:42:08 CEST
*** Bug 6764 has been marked as a duplicate of this bug. ***

CC: (none) => thierry.vignaud

Manuel Hiebel 2013-07-03 10:01:12 CEST

CC: (none) => boklm

Comment 16 Dan Fandrich 2013-07-04 09:33:29 CEST
This has been fixed in version 1.10.5 in Cauldron.
Comment 17 Nicolas Vigier 2013-07-04 10:14:47 CEST
Thanks !
Comment 18 Sandro CAZZANIGA 2013-07-04 10:20:19 CEST
Can we backport the fix in MGA 2-3 ?

CC: (none) => cazzaniga.sandro

Comment 19 Dan Fandrich 2013-07-04 23:21:39 CEST
Sure, I can do that. Technically, the upgrade policy says we should be just patching the current version of mgarepo for MGA 2-3. But, the changes to mgarepo since 1.10.2 have been exclusively bug fixes, with two exceptions: adding bash-completion and allow keeping old rpm log in package directory. I've been running the latest version on mga2 without an issue, and given that the audience for mgarepo is exclusively us, I hope we won't mind bending the rules ever so slightly in this case. Unless I hear objections, I'll back port the latest version to 2 & 3.
Dan Fandrich 2013-07-04 23:21:58 CEST

Assignee: bugsquad => dan

Comment 20 Dave Hodgins 2013-07-05 01:00:07 CEST
(In reply to Dan Fandrich from comment #19)
> Sure, I can do that. Technically, the upgrade policy says we should be just
> patching the current version of mgarepo for MGA 2-3. But, the changes to
> mgarepo since 1.10.2 have been exclusively bug fixes, with two exceptions:
> adding bash-completion and allow keeping old rpm log in package directory.
> I've been running the latest version on mga2 without an issue, and given
> that the audience for mgarepo is exclusively us, I hope we won't mind
> bending the rules ever so slightly in this case. Unless I hear objections,
> I'll back port the latest version to 2 & 3.

Fine with me. Since the upstream doesn't support older
versions :-), it fit's into the exception of allowing
updates to new versions, to be used for bug fixes.

CC: (none) => davidwhodgins

Comment 21 Sandro CAZZANIGA 2013-07-05 06:51:36 CEST
(In reply to Dan Fandrich from comment #19)
> Sure, I can do that. Technically, the upgrade policy says we should be just
> patching the current version of mgarepo for MGA 2-3. But, the changes to
> mgarepo since 1.10.2 have been exclusively bug fixes, with two exceptions:
> adding bash-completion and allow keeping old rpm log in package directory.
> I've been running the latest version on mga2 without an issue, and given
> that the audience for mgarepo is exclusively us, I hope we won't mind
> bending the rules ever so slightly in this case. Unless I hear objections,
> I'll back port the latest version to 2 & 3.

Thanks a lot :)
Comment 22 Dan Fandrich 2013-07-06 00:24:25 CEST
I have uploaded an updated package for Mageia 2 and 3. This update also fixes open bug #4053.

You can test the major issue with the commands:

mkdir tst; cd tst; mgarepo co null; cd null; svn up -rr445770; sed -i 's/Release/#Release/' SPECS/null.spec; mgarepo up

The result should be the message "Conflict discovered" and a menu where you should be able to type tc<Enter> after which the program should exit.

Suggested advisory for MGA2:
========================
Updated mgarepo package to fix the following issues:

mgarepo would apparently hang if a merge conflict were encountered while performing an "mgarepo up" command (mga#952).

The help text for the putsrpm -l option fixed (mga#4053)

Documented putsrpm in the man page (mga#4055)

Fixed the regexp used to escape rpm macros

Added an additional Requires:
========================

Updated packages in core/updates_testing:
========================
mgarepo-1.10.5-1.mga2

Source RPMs:
mgarepo-1.10.5-1.mga2.src.rpm



========================


Suggested advisory for MGA3:
========================
This update fixed an issue whereby mgarepo would apparently hang if a merge conflict were encountered while performing an "mgarepo up" command (mga#952).

Updated packages in core/updates_testing:
========================
mgarepo-1.10.5-1.mga2

Source RPMs:
mgarepo-1.10.5-1.mga2.src.rpm

Assignee: dan => qa-bugs

Dan Fandrich 2013-07-06 00:25:15 CEST

Whiteboard: MGA1TOO, MGA2TOO => MGA1TOO, MGA2TOO, MGA3TOO

Dan Fandrich 2013-07-06 00:46:54 CEST

Whiteboard: MGA1TOO, MGA2TOO, MGA3TOO => MGA2TOO MGA3TOO has_procedure

Comment 23 claire robinson 2013-07-06 09:55:58 CEST
Well done Dan. Congratulations on your first update :)

Version: Cauldron => 3
Whiteboard: MGA2TOO MGA3TOO has_procedure => MGA2TOO has_procedure

Comment 24 martyn vidler 2013-07-06 11:07:27 CEST
Tested MGA2 64

Installed Mgarepo-1.10.2-1 Had to edit /etc/mgarepo.conf file to allow
## uncomment it in case you don't have a account in the Mageia build system:
mirror = svn://svn.mageia.org/svn/packages/

ran tests as comment 22 output was

Updating '.':
U    SPECS/null.spec
D    SOURCES/test.txt
D    SOURCES/null-10-mga-test-patch.patch
Updated to revision 445770.
Updating '.':

urpme mgarepo

installed mgarepo-1.10.5-1.mga2.noarch.rpm

Ran same tests same output
nothing hung program exited

CC: (none) => martynvidler

Comment 25 martyn vidler 2013-07-06 17:40:26 CEST
Tested MGA3 64

uprmi mgarepo-1.10.3-3.mga3
FYI edit /etc/mgarepo.conf again

null$ svn up -rr445770; sed -i 's/Release/#Release/' SPECS/null.spec; mgarepo up
Updating '.':
U    SPECS/null.spec
D    SOURCES/test.txt
D    SOURCES/null-10-mga-test-patch.patch
Updated to revision 445770.
Updating '.':
Conflict discovered in '/home/spiky/tst/null/SPECS/null.spec'.

program hung Ctrl+c to esc



urpmi mgarepo-1.10.5-1.mga3

tst$
mgarepo co null
Using the svn mirror.
To be able to commit changes, use 'mgarepo switch' first.
A    null/SPECS
A    null/SPECS/null.spec
A    null/SOURCES
A    null/SOURCES/sha1.lst
A    null/SOURCES/test.txt
A    null/SOURCES/null-10-mga-test-patch.patch
 U   null
Checked out revision 450759.
--2013-07-06 17:26:49--  http://binrepo.mageia.org//683d068e55042547e475afa585523edabba7ad33
Resolving binrepo.mageia.org (binrepo.mageia.org)... 212.85.158.147, 2a02:2178:2:7::3
Connecting to binrepo.mageia.org (binrepo.mageia.org)|212.85.158.147|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 112 [application/x-tar]
Saving to: ânull/SOURCES/null-10.tgzâ

100%[====================================================================================================================>] 112         --.-K/s   in 0s

2013-07-06 17:26:49 (3.35 MB/s) - ânull/SOURCES/null-10.tgzâ saved [112/112]

null$ 
svn up -rr445770; sed -i 's/Release/#Release/' SPECS/null.spec; mgarepo up
Updating '.':
U    SPECS/null.spec
D    SOURCES/test.txt
D    SOURCES/null-10-mga-test-patch.patch
Updated to revision 445770.
Updating '.':
Conflict discovered in '/home/spiky/tst/null/SPECS/null.spec'.
Select: (p) postpone, (df) diff-full, (e) edit,
        (mc) mine-conflict, (tc) theirs-conflict,
        (s) show all options: tc <<typed
G    SPECS/null.spec
A    SOURCES/test.txt
A    SOURCES/null-10-mga-test-patch.patch
Updated to revision 450760.

Dan Did I run this correct if not pls point out
Comment 26 Dan Fandrich 2013-07-06 21:25:11 CEST
martyn, your test sequence for MGA3 64 looks perfect to me! I left out an unstated assumption in the test steps, namely that mgarepo must first be configured properly to check out packages, which is why that /etc/mgarepo.conf edit was necessary for you.

Your MGA2 64 test log isn't quite as clear. Did the run with 1.10.2 end with the program hanging after Updating '.': the same way it did with the MGA3 test run? Did the run with 1.10.5 say "Conflict discovered..." etc. in the same way as the MGA3 run?

Version: 3 => Cauldron

Comment 27 martyn vidler 2013-07-06 21:39:17 CEST
Dan

It returned me to the prompt after the 
$ mgarepo co null
Using the svn mirror.
To be able to commit changes, use 'mgarepo switch' first.
A    null/SPECS
--2013-07-06 10:21:43--  http://binrepo.mageia.org//683d068e55042547e475afa585523edabba7ad33
Resolving binrepo.mageia.org (binrepo.mageia.org)... 212.85.158.147, 2a02:2178:2:7::3
Connecting to binrepo.mageia.org (binrepo.mageia.org)|212.85.158.147|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 112 [application/x-tar]
Saving to: `null/SOURCES/null-10.tgz'

100%[=======================================================================================================================================>] 112         --.-K/s   in 0s

2013-07-06 10:21:43 (3.89 MB/s) - `null/SOURCES/null-10.tgz' saved [112/112

cd null$

svn up -rr445770; sed -i 's/Release /#Release/' SPECS/null.spec; mgarepo up

Updating '.':
U    SPECS/null.spec
D    SOURCES/test.txt
D    SOURCES/null-10-mga-test-patch.patch
Updated to revision 445770.
Updating '.':

FYI The above was copied from terminal
No it never hung
Comment 28 Dan Fandrich 2013-07-06 22:08:36 CEST
This bug has an element of indeterministic timing about it, so it's not too surprising that you're seeing some slightly different results. Comment #1 and comment #2 both showed other slightly different manifestations of this bug. If you want to double check, run "sha1sum SPECS/null.spec" when the final mgarepo up step completes. A working mgarepo will show

7171456019c92949afdcc3d11c6cb0133b7f8bdc  SPECS/null.spec

A buggy mgarepo will show something else.
claire robinson 2013-07-06 23:50:46 CEST

Version: Cauldron => 3

Comment 29 martyn vidler 2013-07-07 09:10:59 CEST
Dan

sha1sum

mga2 64
SPECS$ sha1sum null.spec
7171456019c92949afdcc3d11c6cb0133b7f8bdc  null.spec

mga3 64
SPECS$ sha1sum null.spec
7171456019c92949afdcc3d11c6cb0133b7f8bdc  null.spec
Comment 30 martyn vidler 2013-07-07 11:20:50 CEST
Tested MGA2 32

Ran the same test as comment 25

I had the same error as above comment 25 with the program hanging Ctrl+c to return to prompt

Updated again the same as as comment 25 

sha1sum 7171456019c92949afdcc3d11c6cb0133b7f8bdc  null.spec

Dan I did copy all of the terminal commands & results into a txt file, if you need anything.
Comment 31 martyn vidler 2013-07-07 11:54:13 CEST
MGA3 32

Test runs the same as above hangs before update

Then having to enter "tc" after 2nd command

sha1sum

7171456019c92949afdcc3d11c6cb0133b7f8bdc  null.spec
Comment 32 Dan Fandrich 2013-07-08 08:22:31 CEST
Did the hang you mention in comment #30 with 1.10.3 or 1.10.5? You're testing so many combinations here, it's not always clear which output corresponds to which test case. The important thing is that 1.10.5 always gives the right sha1sum.

Version: 3 => Cauldron

Comment 33 claire robinson 2013-07-08 08:25:44 CEST
Could you leave this update request set to Mageia 3 Dan please.

Version: Cauldron => 3

Comment 34 Dan Fandrich 2013-07-08 09:09:41 CEST
Sorry--I'm not deliberately changing anything, the bug is loading up with Cauldron always selected for me.
Sandro CAZZANIGA 2013-07-08 09:14:35 CEST

CC: cazzaniga.sandro => (none)

Comment 35 martyn vidler 2013-07-08 18:17:14 CEST
Hi Dan

Sorry for confusion it only hung on the Earlier version. After upgrading to latest
it did not hang on any arch.
martyn vidler 2013-07-08 18:25:38 CEST

Whiteboard: MGA2TOO has_procedure => MGA2TOO has_procedure MGA2-32-ok MGA2-62-ok MGA3-32-ok MGA3-64-ok

martyn vidler 2013-07-08 18:26:39 CEST

Whiteboard: MGA2TOO has_procedure MGA2-32-ok MGA2-62-ok MGA3-32-ok MGA3-64-ok => MGA2TOO has_procedure MGA2-32-ok MGA2-64-ok MGA3-32-ok MGA3-64-ok

Comment 36 martyn vidler 2013-07-08 21:41:34 CEST
validating mgarepo

Advisory and sprms comment 22
Comment 37 Dave Hodgins 2013-07-09 04:32:34 CEST
http://svnweb.mageia.org/advisories/952.mga2.adv?sortby=date&view=log and
http://svnweb.mageia.org/advisories/952.mga3.adv?sortby=date&view=log
uploaded to svn. Could someone from the sysadmin team push 952.mga2.adv and
952.mga3.adv

Keywords: (none) => validated_update
CC: (none) => sysadmin-bugs

Comment 38 Thomas Backlund 2013-07-09 21:35:27 CEST
mga2 update pushed:
http://advisories.mageia.org/MGAA-2013-0052.html

mga3 update pushed:
http://advisories.mageia.org/MGAA-2013-0053.html

Status: NEW => RESOLVED
CC: (none) => tmb
Resolution: (none) => FIXED

Comment 39 Mageia Robot 2013-12-17 19:29:42 CET
commit 183ff56dde67dae60620c880a4b09c867a0c32cd
Author: Dan Fandrich <danf@...>
Date:   Tue Jul 2 21:14:13 2013 +0000

    Don't block waiting for stderr when displaying subprocess output (mga#952)
---
 Commit Link:
   http://gitweb.mageia.org/software/build-system/mgarepo/commit/?id=183ff56dde67dae60620c880a4b09c867a0c32cd
Nicolas Vigier 2014-05-08 18:07:18 CEST

CC: boklm => (none)


Note You need to log in before you can comment on or make changes to this bug.