Bug 32373

Summary: no networking after switch to NetworkManager and reboot
Product: Mageia Reporter: Pierre Fortin <pfortin>
Component: RPM PackagesAssignee: Mageia Bug Squad <bugsquad>
Status: NEW --- QA Contact:
Severity: normal    
Priority: Normal CC: ftg, lewyssmith, marja11
Version: Cauldron   
Target Milestone: ---   
Hardware: All   
OS: Linux   
See Also: https://bugs.mageia.org/show_bug.cgi?id=32409
Whiteboard:
Source RPM: CVE:
Status comment:
Attachments: jounal output while configureing wifi via mcc

Description Pierre Fortin 2023-10-11 11:06:23 CEST
Description of problem: rebooted system to restore /tmp (see bug 32371).
Part of boot took 5m15s reporting:
  Job network-up.service/start running  (xxx / 5min 15s)  <--- OVER 5 MINUTES!!
When the system finished booting; no WiFi.  Trying to bring it up via mcc does not work.

journalctl -b reports:
Oct 11 02:57:55 sensord[4095]: Chip: iwlwifi_1-virtual-0
Oct 11 02:57:55 sensord[4095]: Error getting sensor data: iwlwifi_1/#0: Can't read

dmesg:
[   10.933089] Intel(R) Wireless WiFi driver for Linux
[   10.933660] iwlwifi 0000:0a:00.0: enabling device (0000 -> 0002)
[   10.954768] iwlwifi 0000:0a:00.0: Detected crf-id 0x400410, cnv-id 0x400410 wfpm id 0x80000000
[   10.954907] iwlwifi 0000:0a:00.0: PCI dev 2725/1674, rev=0x420, rfid=0x10d000
[   10.965663] iwlwifi 0000:0a:00.0: api flags index 2 larger than supported by driver
[   10.965677] iwlwifi 0000:0a:00.0: TLV_FW_FSEQ_VERSION: FSEQ Version: 0.0.2.41
[   10.966183] iwlwifi 0000:0a:00.0: loaded firmware version 83.e8f84e98.0 ty-a0-gf-a0-83.ucode op_mode iwlmvm
[   11.053149] iwlwifi 0000:0a:00.0: Detected Killer(R) Wi-Fi 6E AX1675x 160MHz Wireless Network Adapter (210NGW), REV=0x420
[   11.060629] iwlwifi 0000:0a:00.0: WRT: Invalid buffer destination
[   11.220732] iwlwifi 0000:0a:00.0: WFPM_UMAC_PD_NOTIFICATION: 0x20
[   11.220780] iwlwifi 0000:0a:00.0: WFPM_LMAC2_PD_NOTIFICATION: 0x1f
[   11.220822] iwlwifi 0000:0a:00.0: WFPM_AUTH_KEY_0: 0x90
[   11.220835] iwlwifi 0000:0a:00.0: CNVI_SCU_SEQ_DATA_DW9: 0x0
[   11.221983] iwlwifi 0000:0a:00.0: loaded PNVM version 181407b3
[   11.237475] iwlwifi 0000:0a:00.0: Detected RF GF, rfid=0x10d000
[   11.307973] iwlwifi 0000:0a:00.0: base HW address: 44:e5:17:fd:11:87
[   17.396681] iwlwifi 0000:0a:00.0 wlp10s0: renamed from wlan0

ifconfig does not show the wifi interface, and:

$ ifconfig up wlp10s0
SIOCSIFADDR: No such device
up: ERROR while getting interface flags: No such device

$ systemctl status network.service 
● network.service - LSB: Bring up/down networking
     Loaded: loaded (/etc/rc.d/init.d/network; generated)
     Active: activating (start) since Wed 2023-10-11 03:44:10 EDT; 1min 5s ago
       Docs: man:systemd-sysv-generator(8)
Cntrl PID: 1291757 (network)
      Tasks: 7 (limit: 154182)
     Memory: 102.8M
        CPU: 1min 55.848s
     CGroup: /system.slice/network.service
             ├─   6190 /sbin/ifplugd -I -b -i p5p1
             ├─1291757 /usr/bin/bash /etc/rc.d/init.d/network start
             ├─1291965 /sbin/ifplugd -I -b -i docker0
             ├─1408169 /usr/bin/bash /etc/sysconfig/network-scripts/ifup-eth ifcfg-veth0afcb3a boot
             ├─1408233 /usr/bin/bash /etc/sysconfig/network-scripts/ifup-eth ifcfg-veth0afcb3a boot
             ├─1408234 modprobe -c
             └─1408236 awk "BEGIN { alias = \"\"; }\n        \$1 == \"alias\" && \$2 == \"veth0afcb3a\" { alias = \$3; }\n        \$1 == \"install\" { install[\$2] = \$3; }\n        END {\n            cmd = install[alias];\>



Oct 11 03:45:16 pf.pfortin.com network[1407229]: ERROR     : [/etc/sysconfig/network-scripts/ifup-eth] Device veth0afb3d1 does not seem to be present, delaying initialization.
Oct 11 03:45:16 pf.pfortin.com network[1291757]: [FAILED]
Oct 11 03:45:16 pf.pfortin.com network[1291757]: Bringing up interface veth0afb84a:
Oct 11 03:45:16 pf.pfortin.com network[1407550]: ERROR     : [/etc/sysconfig/network-scripts/ifup-eth] Device veth0afb84a does not seem to be present, delaying initialization.
Oct 11 03:45:16 pf.pfortin.com /etc/sysconfig/network-scripts/ifup-eth[1407822]: Device veth0afb84a does not seem to be present, delaying initialization.
Oct 11 03:45:16 pf.pfortin.com network[1291757]: [FAILED]
Oct 11 03:45:16 pf.pfortin.com network[1291757]: Bringing up interface veth0afc95f:
Oct 11 03:45:16 pf.pfortin.com network[1407852]: ERROR     : [/etc/sysconfig/network-scripts/ifup-eth] Device veth0afc95f does not seem to be present, delaying initialization.
Oct 11 03:45:16 pf.pfortin.com /etc/sysconfig/network-scripts/ifup-eth[1408139]: Device veth0afc95f does not seem to be present, delaying initialization.
Oct 11 03:45:16 pf.pfortin.com network[1291757]: [FAILED]

Couple reboots later; no change. WiFi will not come up.  It was working fine just before reboot.


May be related to one or more of these updates:
$ rpm -qa --last | grep -i net
python3-qt6-networkauth-6.5.0-1.mga10.x86_64  Wed 11 Oct 2023 02:19:19 AM EDT
lib64qt6network-devel-6.5.2-2.mga10.x86_64    Wed 11 Oct 2023 02:10:47 AM EDT
lib64qt6network6-6.5.2-2.mga10.x86_64         Wed 11 Oct 2023 02:10:47 AM EDT
lib64qt5network-devel-5.15.7-8.mga10.x86_64   Wed 11 Oct 2023 02:10:47 AM EDT
lib64qt5network5-5.15.7-8.mga10.x86_64        Wed 11 Oct 2023 02:10:47 AM EDT

My last reboots:
Fri Sep 29 11:25:49 PM EDT 2023
Mageia release 10 (Cauldron) for x86_64
Kernel 6.5.3-server-1.mga10 on a 20-processor x86_64 / \l
Linux pf.pfortin.com 6.5.3-server-1.mga10 #1 SMP PREEMPT_DYNAMIC Sat Sep 16 00:04:28 UTC 2023 x86_64 GNU/Linux

Fri Sep 29 11:35:01 PM EDT 2023
Mageia release 10 (Cauldron) for x86_64
Kernel 6.5.3-server-1.mga10 on a 20-processor x86_64 / \l
Linux pf.pfortin.com 6.5.3-server-1.mga10 #1 SMP PREEMPT_DYNAMIC Sat Sep 16 00:04:28 UTC 2023 x86_64 GNU/Linux

Wed Oct 11 02:34:24 AM EDT 2023
Mageia release 10 (Cauldron) for x86_64
Kernel 6.5.3-server-1.mga10 on a 20-processor x86_64 / \l
Linux pf.pfortin.com 6.5.3-server-1.mga10 #1 SMP PREEMPT_DYNAMIC Sat Sep 16 00:04:28 UTC 2023 x86_64 GNU/Linux

Wed Oct 11 03:03:34 AM EDT 2023
Mageia release 10 (Cauldron) for x86_64
Kernel 6.5.3-server-1.mga10 on a 20-processor x86_64 / \l
Linux pf.pfortin.com 6.5.3-server-1.mga10 #1 SMP PREEMPT_DYNAMIC Sat Sep 16 00:04:28 UTC 2023 x86_64 GNU/Linux

Running on ethernet:
5p1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.47  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::7686:e2ff:fe14:833d  prefixlen 64  scopeid 0x20<link>
        ether 74:86:e2:14:83:3d  txqueuelen 1000  (Ethernet)
        RX packets 662001  bytes 825652192 (787.4 MiB)
        RX errors 0  dropped 3  overruns 0  frame 0
        TX packets 392976  bytes 135223244 (128.9 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


How reproducible: always


Steps to Reproduce:
1.
2.
3.
Comment 1 Pierre Fortin 2023-10-14 03:49:10 CEST
Besides this issue with my main system, a laptop also will not work over wifi.
Using mcc to (try to) configure wifi, they "see' the various AP SSIDs; but won't connect.

What killed wifi?
Comment 2 Pierre Fortin 2023-10-15 01:08:48 CEST
Getting more interesting...  I have a USB wifi adapter from an old RasPi system; that works...  So, built-in devices quit working; but the issue does not affect this USB device. I REALLY need the laptop for an upcoming meeting, so I was rather irritated until I recalled the USB adapter...  it's only 2.4GHz; but it gives me access...

YIKES...  only until a reboot...  then neither adapter works...  If you're not seeing an issue yet, try rebooting...

Documenting as I go...

Checking recent changes in /etc...

/etc/modprobe.conf:
remove sdhci_pci /sbin/modprobe -r mmc_block; /sbin/modprobe --first-time -r --ignore-remove sdhci_pci
install sdhci_pci /sbin/modprobe --first-time --ignore-install sdhci_pci && { /sbin/modprobe mmc_block; /bin/true; }
remove rtsx_pci /sbin/modprobe -r mmc_block; /sbin/modprobe --first-time -r --ignore-remove rtsx_pci
install rtsx_pci /sbin/modprobe --first-time --ignore-install rtsx_pci && { /sbin/modprobe mmc_block; /bin/true; }
options snd-ac97-codec power_save=1
alias enp1s0f1 r8169
alias sound-slot-1 snd_hda_intel
alias sound-slot-0 snd_hda_intel
install sdhci /sbin/modprobe --first-time --ignore-install sdhci && { /sbin/modprobe mmc_block; /bin/true; }
remove sdhci /sbin/modprobe -r mmc_block; /sbin/modprobe --first-time -r --ignore-remove sdhci
install usb-interface /sbin/modprobe ehci_hcd; /sbin/modprobe ehci_pci; /sbin/modprobe xhci_hcd; /sbin/modprobe xhci_pci_renesas; /sbin/modprobe xhci_pci; /bin/true
alias wlp2s0 ath9k
alias pci:v0000168Cd00000036sv000011ADsd00000642bc02sc80i00 ath9k
alias wlp0s16u1 rt2800usb

Commented out the last 3 lines and rebooted. WiFi still didn't come up.

Last 3 lines of /etc/modprobe.conf are now
#alias pci:v0000168Cd00000036sv000011ADsd00000642bc02sc80i00 ath9k
#alias wlp0s16u1 rt2800usb
alias wlp0s16u1 rt2800usb   # got uncommented

Tryin again to setup wifi, getting this on konsole:
Selected interface 'wlp0s16u1'
OK
WARN      : [ifdown] You are using 'ifdown' script provided by 'network-scripts', which are now deprecated.
WARN      : [ifdown] 'network-scripts' will be removed from distribution in near future.
WARN      : [ifdown] It is advised to switch to 'NetworkManager' instead - it provides 'ifup/ifdown' scripts as well.
WARN      : [ifup] You are using 'ifup' script provided by 'network-scripts', which are now deprecated.
WARN      : [ifup] 'network-scripts' will be removed from distribution in near future.
WARN      : [ifup] It is advised to switch to 'NetworkManager' instead - it provides 'ifup/ifdown' scripts as well.
Error for wireless request "Set Encode" (8B2A) :
    SET failed on device wlp0s16u1 ; Invalid argument.
Successfully initialized wpa_supplicant
Line 2079: unknown network field 'wep_key0'.      <--- say WHAT??
Line 2079: unsupported WEP parameter              <--- it's ALWAYS been 
Line 2081: unknown network field 'wep_tx_keyidx'. <--- WPA/WPA2 Pre-shared key
Line 2081: unsupported WEP parameter              <---
ioctl[SIOCSIWENCODEEXT]: Invalid argument
ioctl[SIOCSIWENCODEEXT]: Invalid argument
Selected interface 'wlp0s16u1'
OK
Comment 3 Pierre Fortin 2023-10-15 01:11:24 CEST
Created attachment 14057 [details]
jounal output while configureing wifi via mcc

With a clue from laptop testing; tried configuring wifi on main system.  wifi adapter "sees" SSIDs but won't connect..
Comment 4 Pierre Fortin 2023-10-18 17:38:15 CEST
Checking journal, it recommends switching to NetworkManager...

Following procedure at https://wiki.mageia.org/en/Switching_to_networkmanager shows all NM packages were already installed. Everything seemed to work as specified.
However, the final step: systemctl mask network.service; systemctl mask network-up
returned:
Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service → /usr/lib/systemd/system/NetworkManager.service.
Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service → /usr/lib/systemd/system/NetworkManager-dispatcher.service.
Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service → /usr/lib/systemd/system/NetworkManager-wait-online.service.

[Command not returning...]

^Z
[11]+  Stopped                 systemctl enable --now NetworkManager.service

# top | grep Network
 197330 root      20   0  369640  52100  17540 R 100.0   0.0   1:34.45 NetworkManager                                                                                                                                           
 197330 root      20   0  370640  53060  17540 R 100.0   0.0   1:37.46 NetworkManager  
q
                                                                                                                                         
# ps aux | grep NetworkManager
root      197330 99.8  0.0 442396 124420 ?       RLsl 11:19   5:00 /usr/sbin/NetworkManager --no-daemon
# kill -9 197330
[ it just restarts at 100% ]
# ps aux | grep NetworkManager
root      198454  104  0.0 337180 20628 ?        RLsl 11:24   0:01 /usr/sbin/NetworkManager --no-daemon

This doesn't seem right...

Will reboot to see if NM still eats CPU...
Pierre Fortin 2023-10-18 20:23:47 CEST

Summary: no WiFi after reboot => no networking after switch to NetworkManager and reboot

Comment 5 Pierre Fortin 2023-10-18 21:07:57 CEST
Yes, NM still runs 100% after reboot; AND has killed ALL but "lo" interfaces. To add insult to injury, NM takes 10+ minutes to startup.

Starting mcc to setup networking, mcc just hangs on "Loading...Please wait"...

I was able to restore at least ethernet with:
Changing .net_applet back to TRUE
systemctl unmask network.service
Comment 6 Frank Griffin 2023-10-19 00:45:57 CEST
I can't comment on the mixture of packages, since I always run full cauldron, but I've never had a problem with NM and ethernet on one desktop or NM and WiFi on a succession of laptops over the years.

There are some blips.  Originally, NM had two states for an SSID: enable for all users, in which case connection would be made at NM initialization, or enable for a specific user, in which case connection would be made when that user logged in.  That seems to be gone now, because NM only connects now under Plasma when a user logs in.

After a clean installation, I generally have to enable and start NM because we still don't do this by default.  Then, before I start a DE, I use nmtui from a virtual console to activate the interfaces I want before starting the DE.  For Plasma, you need to have installed the Plasma NM applet.  GNOME includes NM support by default.

Either NM or the applet used to cache SSID passwords, but now Plasma appears to want to use kdewallet, and if you decline it wants you to re-authenticate every time the DE starts, but the connection always succeeds.

This behavior has persisted across several laptops with different Wifi chipsets.  My cauldron installs use custom desktops with all options enabled.

CC: (none) => ftg

Marja Van Waes 2023-10-25 22:06:04 CEST

See Also: (none) => https://bugs.mageia.org/show_bug.cgi?id=32409
CC: (none) => marja11

Comment 7 Pierre Fortin 2023-10-31 00:31:45 CET
Found what may be cause for slow boot...

/etc/sysconfig/network-scripts  contained 75,296 files, named ifcfg-veth*, such as: ifcfg-veth22166f4, ifcfg-veth75397e9, ifcfg-veth68086ae, etc...
All containing:
DEVICE=veth68086ae  <- matches file name
BOOTPROTO=dhcp
ONBOOT=yes

The file dates are Sep 29 thru Oct 11 which fall between these reboots:
Fri Sep 29 11:35:01 PM EDT 2023
Mageia release 10 (Cauldron) for x86_64
Kernel 6.5.3-server-1.mga10 on a 20-processor x86_64 / \l
Linux pf.pfortin.com 6.5.3-server-1.mga10 #1 SMP PREEMPT_DYNAMIC Sat Sep 16 00:04:28 UTC 2023 x86_64 GNU/Linux

Wed Oct 11 02:34:24 AM EDT 2023
Mageia release 10 (Cauldron) for x86_64
Kernel 6.5.3-server-1.mga10 on a 20-processor x86_64 / \l
Linux pf.pfortin.com 6.5.3-server-1.mga10 #1 SMP PREEMPT_DYNAMIC Sat Sep 16 00:04:28 UTC 2023 x86_64 GNU/Linux

For the most part, they were created at _random_ times; though on average about 4 per minute most of the time; some repeated within one second -- no repeating time pattern that I could see. 

I have not rebooted since Sat Oct 28 09:02:59 AM EDT 2023 and I won't have time to do anything with this before Tue Nov 7.
Comment 8 Lewis Smith 2023-11-03 12:53:58 CET
This last information is good - well done for spotting it.
Given the change in the direction of this bug from comments 3 to 4, from then on it looks like its newer related bug  32409.

CC: (none) => lewyssmith