Bug 34144 - wapiti and some of its requires need to be updated
Summary: wapiti and some of its requires need to be updated
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: 9
Hardware: All Linux
Priority: Normal normal
Target Milestone: ---
Assignee: QA Team
QA Contact:
URL:
Whiteboard: MGA9-64-OK
Keywords: advisory, validated_update
: 34021 (view as bug list)
Depends on:
Blocks:
 
Reported: 2025-03-30 18:55 CEST by katnatek
Modified: 2025-04-05 20:47 CEST (History)
6 users (show)

See Also:
Source RPM: wapiti-3.0.5-3.mga9,python-browser-cookie3-0.12.1-3.mga9,python-httpx-0.23.0-1.mga9
CVE:
Status comment: Package list in comment #17


Attachments
Error message in terminal for wapiti 3.1.4 (7.25 KB, text/plain)
2025-04-02 03:57 CEST, katnatek
Details
Log of the update from current version (14.95 KB, text/plain)
2025-04-04 22:47 CEST, katnatek
Details
Log of clean install current version (21.57 KB, text/plain)
2025-04-04 22:56 CEST, katnatek
Details

Description katnatek 2025-03-30 18:55:35 CEST
Description of problem: While attempting to test a python3-zipp update, I installed wapiti and dependencies, including python3-zipp, in a VirtualBox Plasma guest. The installation had no issues, but when I ran the most basic of commands, I got this:

$ wapiti -h
Traceback (most recent call last):
  File "/usr/bin/wapiti", line 33, in <module>
    sys.exit(load_entry_point('wapiti3==3.0.5', 'console_scripts', 'wapiti')())
  File "/usr/bin/wapiti", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/lib64/python3.10/importlib/metadata/__init__.py", line 171, in load
    module = import_module(match.group('module'))
  File "/usr/lib64/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/lib/python3.10/site-packages/wapitiCore/main/wapiti.py", line 39, in <module>
    import browser_cookie3
  File "/usr/lib/python3.10/site-packages/browser_cookie3/__init__.py", line 10, in <module>
    import lz4.block
ModuleNotFoundError: No module named 'lz4'

I was informed that 'lz4' is provided by 'python3-lz4' so I installed that and tried again:

$ wapiti -h
Traceback (most recent call last):
  File "/usr/bin/wapiti", line 33, in <module>
    sys.exit(load_entry_point('wapiti3==3.0.5', 'console_scripts', 'wapiti')())
  File "/usr/bin/wapiti", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/lib64/python3.10/importlib/metadata/__init__.py", line 171, in load
    module = import_module(match.group('module'))
  File "/usr/lib64/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/lib/python3.10/site-packages/wapitiCore/main/wapiti.py", line 39, in <module>
    import browser_cookie3
  File "/usr/lib/python3.10/site-packages/browser_cookie3/__init__.py", line 14, in <module>
    from Crypto.Cipher import AES
ModuleNotFoundError: No module named 'Crypto'

That makes at least two missing dependencies. That's as far as I've gone, so there may be more.

It was detected tha some of its requires need to be updated
Comment 1 katnatek 2025-03-30 18:56:57 CEST
*** Bug 34021 has been marked as a duplicate of this bug. ***
Comment 2 papoteur 2025-03-31 17:34:21 CEST
I don't understand why you opened a new report. The topic is still the same.
It seems wapiti never worked in Mageia 9. It requires httpx in exactly 0.17 release, when we have 0.23.0 currently. This version have modified some calls in between and wapiti 3.05 is not compatible with httpx-0.23.0
wapiti 3.1.4 claims exactly httpx-0.23.0, but it needs to also import:
        "aiocache==0.11.1",
        "aiosqlite==0.17.0",
        "loguru>=0.5.3",
        "arsenic==21.8"


I'm reluctant to update python-httpx because it seems to often break API even with minor updates, and it is required by other packages.
Comment 3 katnatek 2025-03-31 18:03:47 CEST
(In reply to papoteur from comment #2)
> I don't understand why you opened a new report. The topic is still the same.
> It seems wapiti never worked in Mageia 9. It requires httpx in exactly 0.17
> release, when we have 0.23.0 currently. This version have modified some
> calls in between and wapiti 3.05 is not compatible with httpx-0.23.0
> wapiti 3.1.4 claims exactly httpx-0.23.0, but it needs to also import:
>         "aiocache==0.11.1",
>         "aiosqlite==0.17.0",
>         "loguru>=0.5.3",
>         "arsenic==21.8"
> 
> 
> I'm reluctant to update python-httpx because it seems to often break API
> even with minor updates, and it is required by other packages.

It's the same but is too long right now and mix some other things, yes I now that, even try to update to 3.0.7 needs some extra things

BTW what to do with the other packages ?

python3-httpcore-1.0.7-1.mga9
python3-httpx-socks-0.10.0-1.mga9
python3-browser-cookie3-0.20.1-1.mga9 This looks the only safe for update

python3-socksio-1.0.0-1.1.mga9

NS for update
python3-httpx+brotli-0.28.1-1.1.mga9
python3-httpx+cli-0.28.1-1.1.mga9
python3-httpx+http2-0.28.1-1.1.mga9
python3-httpx+socks-0.28.1-1.1.mga9
python3-httpx-0.28.1-1.1.mga9
Comment 4 papoteur 2025-03-31 18:17:03 CEST
urpmq --whatrequires python3-httpcore
python3-httpcore
python3-httpx
python3-httpx-socks

Thus, no need to update httpcore (except if it is broken with current httpx)

urpmq --whatrequires python3-httpx-socks
python3-httpx-socks
wapiti
Thus it can follow what wapiti needs.
Comment 5 katnatek 2025-03-31 18:44:07 CEST
Well we are in dead end and then this should be closed as wontfix

(In reply to Dan Fandrich from bug#34021 comment #11)
> It looks like httpcore.SyncConnectionPool() was removed from httpcore 4
> years ago in version 0.14 with this commit
> https://github.com/encode/httpcore/commit/
> f9b93918a54a49a4e917824ad38cf5bd8da21450
> 
> I guess httpx_socks needs to be updated to handle that change.


https://github.com/wapiti-scanner/wapiti/blob/3.0.5/setup.py
httpx==0.17.1 

So our httpx is newer than needed :P

And update wapiti even to next version could be a headche https://github.com/wapiti-scanner/wapiti/blob/3.0.7/setup.py

bug#34021 comment#13
After build that version and install packages that I build and needed by this version, still needs

python3dist(httpx[brotli])

In resume if update httpcore need to update to compatible httpx 
to keep them in sync

Just update browsercookie and socks not helps

from httpcore._backends.sync import SyncStream as CoreSyncStream
ModuleNotFoundError: No module named 'httpcore._backends'
Comment 6 katnatek 2025-03-31 23:59:22 CEST
(In reply to papoteur from comment #2)
> I don't understand why you opened a new report. The topic is still the same.
> It seems wapiti never worked in Mageia 9. It requires httpx in exactly 0.17
> release, when we have 0.23.0 currently. This version have modified some
> calls in between and wapiti 3.05 is not compatible with httpx-0.23.0
> wapiti 3.1.4 claims exactly httpx-0.23.0, but it needs to also import:
>         "aiocache==0.11.1",
>         "aiosqlite==0.17.0",
>         "loguru>=0.5.3",
>         "arsenic==21.8"
> 
> 
> I'm reluctant to update python-httpx because it seems to often break API
> even with minor updates, and it is required by other packages.

urpmq --whatrequires-recursive python3-httpx|uniq

mnemosyne
orange
python3-aioruuvigateway
python3-googletrans
python3-httpx
python3-httpx-socks
wapiti

But
urpmq --requires mnemosyne
python(abi)[== 3.10]
python3-argon2-cffi
python3-cherrypy
python3-googletrans
python3-gtts
python3-matplotlib-qt5
python3-qt5-core
python3-qt5-gui
python3-qt5-sql
python3-qt5-webenginewidgets
python3-qt5-widgets
python3-webob

So even if affect mnemosyne some function is not working due python3-googletrans

I can't confirm orange because core dumps in my system current or updated packages

I have ready
>         "aiocache==0.12.3",
>         "aiosqlite==0.20.0",
>         "loguru==0.7.3",
>         "arsenic==21.8"
As part of work to try update wapiti to latest version in cauldron, but I don't
know if wapiti 3.1.4 will like those versions

I'll make some dig in my other copr
Comment 7 katnatek 2025-04-01 00:18:46 CEST
But due httpx required by latest python3-googletrans ,looks like jumps to wapiti 3.2.0 could be required if his httpx 0.27.0 requirement is not sine qua non
But also requires prance that is giving my some headche

In resume, more motives to not update wapiti in mageia 9
Comment 8 katnatek 2025-04-01 01:14:06 CEST
Even current httpx is incomplete for wapiti 3.1.4 

No matching package to install: 'python3dist(httpx[brotli]) >= 0.23'
No matching package to install: 'python3dist(httpx[socks]) >= 0.23'

If understand well https://github.com/encode/httpx/blob/0.23.0/setup.py

"socks": "socksio==1.*", requires python3-socksio-1.0.0-1.1.mga9 in testing
Comment 9 katnatek 2025-04-01 05:11:56 CEST
(In reply to katnatek from comment #8)
> Even current httpx is incomplete for wapiti 3.1.4 
> 
> No matching package to install: 'python3dist(httpx[brotli]) >= 0.23'
> No matching package to install: 'python3dist(httpx[socks]) >= 0.23'
> 
> If understand well https://github.com/encode/httpx/blob/0.23.0/setup.py
> 
> "socks": "socksio==1.*", requires python3-socksio-1.0.0-1.1.mga9 in testing

And with all that work still need other thing
ModuleNotFoundError: No module named 'mitmproxy_wireguard'

Project is archived https://github.com/decathorpe/mitmproxy_wireguard/tree/main

XD
Comment 10 katnatek 2025-04-02 03:57:53 CEST
Created attachment 14930 [details]
Error message in terminal for wapiti 3.1.4

It's possible that some of the versions I use to build not likes to this thing
https://copr.fedorainfracloud.org/coprs/katnatek/blogdrake/builds/
Comment 11 katnatek 2025-04-02 04:15:54 CEST
(In reply to katnatek from comment #10)
> Created attachment 14930 [details]
> Error message in terminal for wapiti 3.1.4
> 
> It's possible that some of the versions I use to build not likes to this
> thing
> https://copr.fedorainfracloud.org/coprs/katnatek/blogdrake/builds/

OK I find the solution https://github.com/wapiti-scanner/wapiti/issues/204

wapiti -u http://testhtml5.vulnweb.com/ --flush-session

     __      __               .__  __  .__________
    /  \    /  \_____  ______ |__|/  |_|__\_____  \
    \   \/\/   /\__  \ \____ \|  \   __\  | _(__  <
     \        /  / __ \|  |_> >  ||  | |  |/       \
      \__/\  /  (____  /   __/|__||__| |__/______  /
           \/        \/|__|                      \/
Wapiti 3.1.4 (wapiti-scanner.github.io)
[*] Saving scan state, please wait...
[!] Unable to import module ssl
[!] Unable to find a module named ssl

[*] Launching module xss

[*] Launching module file

[*] Launching module http_headers
Checking X-Frame-Options :
X-Frame-Options is not set
Checking X-Content-Type-Options :
X-Content-Type-Options is not set

[*] Launching module sql

[*] Launching module exec

[*] Launching module ssrf

[*] Launching module cookieflags
Checking cookie : username
HttpOnly flag is not set in the cookie : username
Secure flag is not set in the cookie : username

[*] Launching module csp
CSP is not set

[*] Launching module redirect

[*] Launching module permanentxss

[*] Generating report...
A report has been generated in the file /home/katnatek/.wapiti/generated_report
Open /home/katnatek/.wapiti/generated_report/testhtml5.vulnweb.com_04022025_0210.html with a browser to see this report.

papoteur did you think is worth import all that packages in mageia 9 to fix this?

Dan can you please remove
python3-httpx+brotli-0.28.1-1.1.mga9
python3-httpx+cli-0.28.1-1.1.mga9
python3-httpx+http2-0.28.1-1.1.mga9
python3-httpx+socks-0.28.1-1.1.mga9
python3-httpx-0.28.1-1.1.mga9

From python-httpx-0.28.1-1.1.mga9
Comment 12 katnatek 2025-04-02 04:17:35 CEST
(In reply to katnatek from comment #11)
> Dan can you please remove
> python3-httpx+brotli-0.28.1-1.1.mga9
> python3-httpx+cli-0.28.1-1.1.mga9
> python3-httpx+http2-0.28.1-1.1.mga9
> python3-httpx+socks-0.28.1-1.1.mga9
> python3-httpx-0.28.1-1.1.mga9
> 
> From python-httpx-0.28.1-1.1.mga9

CC: (none) => dan

Comment 13 papoteur 2025-04-02 09:19:26 CEST
(In reply to katnatek from comment #11)

> papoteur did you think is worth import all that packages in mageia 9 to fix
> this?
> 
Great work!
1. httpx is kept in the same version, thus no break on this side.
2. wapiti works now.
3. new packages imported won't break anything
For these reason I would say yes.
What you didn't say is if some packages have to be updated.
Comment 14 katnatek 2025-04-02 10:40:55 CEST
(In reply to papoteur from comment #13)
> (In reply to katnatek from comment #11)
> 
> > papoteur did you think is worth import all that packages in mageia 9 to fix
> > this?
> > 
> Great work!
> 1. httpx is kept in the same version, thus no break on this side.
> 2. wapiti works now.
> 3. new packages imported won't break anything
> For these reason I would say yes.
> What you didn't say is if some packages have to be updated.

Of the testing packages

(In reply to katnatek from comment #3)
> (In reply to papoteur from comment #2)
> python3-httpcore-1.0.7-1.mga9
> python3-httpx-socks-0.10.0-1.mga9

> python3-browser-cookie3-0.20.1-1.mga9
> 
> python3-socksio-1.0.0-1.1.mga9
> 
>
> python3-httpx+brotli-0.28.1-1.1.mga9
> python3-httpx+cli-0.28.1-1.1.mga9
> python3-httpx+http2-0.28.1-1.1.mga9
> python3-httpx+socks-0.28.1-1.1.mga9
> python3-httpx-0.28.1-1.1.mga9

Just use socksio and browser-cokie3

httpx just copy how 0.28.1 version generates the
subpackages

So I guess https-socks and httpcore should
need to be added to remove list

The maturing version is fine? I just
rebuild and fix the check stage of
a.package that create some time ago
Comment 15 katnatek 2025-04-03 00:19:19 CEST
@Dan one more time please remove from 9/core/updates_testing

python3-httpx+brotli-0.28.1-1.1.mga9
python3-httpx+cli-0.28.1-1.1.mga9
python3-httpx+http2-0.28.1-1.1.mga9
python3-httpx+socks-0.28.1-1.1.mga9
python3-httpx-0.28.1-1.1.mga9
python3-httpcore-1.0.7-1.mga9
python3-httpx-socks-0.10.0-1.mga9

From 

python-httpx-0.28.1-1.1.mga9
python-httpcore-1.0.7-1.mga9
python-httpx-socks-0.10.0-1.mga9
Comment 16 Dan Fandrich 2025-04-03 01:09:14 CEST
Those RPMs from comment 15 are now deleted.
katnatek 2025-04-03 05:15:46 CEST

Source RPM: wapiti-3.0.5-3.mga9,python-browser-cookie3-0.12.1-3.mga9,python-httpcore-0.15.0-1.mga9,python-httpx-socks-0.4.1-3.mga9,python-httpx-0.23.0-1.mga9,python-httpcore-0.15.0-1.mga9 => wapiti-3.0.5-3.mga9,python-browser-cookie3-0.12.1-3.mga9,python-httpx-0.23.0-1.mga9
Summary: Some of wapiti's requires need to be updated => wapiti and some of its requires need to be updated

Comment 17 katnatek 2025-04-04 20:24:27 CEST
Packages:

python3-browser-cookie3-0.20.1-1.mga9
python3-socksio-1.0.0-1.1.mga9
python3-httpx+brotli-0.23.0-1.1.mga9
python3-httpx+cli-0.23.0-1.1.mga9
python3-httpx+http2-0.23.0-1.1.mga9
python3-httpx+socks-0.23.0-1.1.mga9
python3-httpx-0.23.0-1.1.mga9
python3-aiomcache-0.8.2-1.mga9
python3-aiosqlite-0.20.0-1.mga9
python-aiocache-doc-0.12.3-1.mga9
python3-aiocache-0.12.3-1.mga9
python3-arsenic-21.8-1.mga9
python3-maturin-1.2.3-1.mga9
python3-mitmproxy-wireguard-0.1.23-1.mga9
python3-loguru-0.5.3-1.mga9
wapiti-3.1.4-1.mga9


SRPMS
python-browser-cookie3-0.20.1-1.mga9
python-socksio-1.0.0-1.1.mga9
python-httpx-0.23.0-1.1.mga9
python-aiomcache-0.8.2-1.mga9
python-aiosqlite-0.20.0-1.mga9
python-aiocache-0.12.3-1.mga9
python-arsenic-21.8-1.mga9
python-maturin-1.2.3-1.mga9
python3-mitmproxy-wireguard-0.1.23-1.mga9
python3-loguru-0.5.3-1.mga9
wapiti-3.1.4-1.mga9

Use the command in comment#11 to test
I not deep test so if you find something still not works please tell

Assignee: j.alberto.vc => qa-bugs
Status comment: (none) => Package list in comment #17

Comment 18 katnatek 2025-04-04 22:47:39 CEST
Created attachment 14935 [details]
Log of the update from current version

wapiti -u http://testhtml5.vulnweb.com/ --flush-session

Works 

Just worry me a few the part
[!] Unable to import module ssl
[!] Unable to find a module named ssl
Comment 19 katnatek 2025-04-04 22:56:15 CEST
Created attachment 14936 [details]
Log of clean install current version

wapiti -u http://testhtml5.vulnweb.com/ --flush-session

Works, this time all the updated and necessary packages are fetched
Comment 20 katnatek 2025-04-04 23:05:58 CEST
orange-canvas from package orange still core dumps for me, I let that test to 
you Thomas
Comment 21 Thomas Andrews 2025-04-05 02:57:58 CEST
Started with the VirtualBox VM where I had the current orange installed, got pending updates, and ran it with the command orange-canvas. The gui came up, and there were no issues running it.

Then I used qarepo to get the packages from comment 17, and once again got any pending updates - this time only python3-httpx. Ran orange-canvas again, with the same results.

Then, I used urpmi to install wapiti. That drew in the remaining packages for comment 17, plus several from other repos. I ran orange-canvas again, and once again the gui came up. This time I played with it a bit, not having a clue about how to use it. It did not crash, but when I closed it there were several messages left in the command line, almost all of which had something to do with a problem with the "keyring."

Ran wapiti with the command from comment 19, and it appeared to work.

Looks like it's OK. If the "keyring" issues seen with orange are an issue, and not just due to user error, it's one for another bug.

katnatek, I think we can send this one on.
Comment 22 Thomas Andrews 2025-04-05 02:59:36 CEST
Validating. Katnatek, it still needs an advisory.

Keywords: (none) => validated_update
CC: (none) => sysadmin-bugs
Whiteboard: (none) => MGA9-64-OK

Comment 23 katnatek 2025-04-05 03:08:19 CEST
(In reply to Thomas Andrews from comment #21)
> Started with the VirtualBox VM where I had the current orange installed, got
> pending updates, and ran it with the command orange-canvas. The gui came up,
> and there were no issues running it.
> 
> Then I used qarepo to get the packages from comment 17, and once again got
> any pending updates - this time only python3-httpx. Ran orange-canvas again,
> with the same results.
> 
> Then, I used urpmi to install wapiti. That drew in the remaining packages
> for comment 17, plus several from other repos. I ran orange-canvas again,
> and once again the gui came up. This time I played with it a bit, not having
> a clue about how to use it. It did not crash, but when I closed it there
> were several messages left in the command line, almost all of which had
> something to do with a problem with the "keyring."
> 
> Ran wapiti with the command from comment 19, and it appeared to work.
> 
> Looks like it's OK. If the "keyring" issues seen with orange are an issue,
> and not just due to user error, it's one for another bug.
> 
Perhaps missing python3-keyring , but will be other issue
katnatek 2025-04-05 03:24:48 CEST

Keywords: (none) => advisory

Comment 24 Mageia Robot 2025-04-05 20:47:46 CEST
An update for this issue has been pushed to the Mageia Updates repository.

https://advisories.mageia.org/MGAA-2025-0035.html

Resolution: (none) => FIXED
Status: NEW => RESOLVED


Note You need to log in before you can comment on or make changes to this bug.