Bug 14904 - Share the Internet connection with other local machines: problems with squid and shorewall
Summary: Share the Internet connection with other local machines: problems with squid ...
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: Cauldron
Hardware: All Linux
Priority: Normal normal
Target Milestone: ---
Assignee: Mageia Bug Squad
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-12-28 22:00 CET by Bjarne Thomsen
Modified: 2015-02-16 20:26 CET (History)
9 users (show)

See Also:
Source RPM: drakx-net
CVE:
Status comment:


Attachments
New internal squid.conf file (1.70 KB, text/plain)
2014-12-28 22:18 CET, Bjarne Thomsen
Details
A minimalistic internal squid.conf based on Mageia5, only. (1.28 KB, text/plain)
2014-12-29 10:20 CET, Bjarne Thomsen
Details
patch to fix #14904 (1.44 KB, patch)
2014-12-30 12:38 CET, AL13N
Details | Diff

Description Bjarne Thomsen 2014-12-28 22:00:57 CET
Description of problem:
After sharing the internet connection:
a) squid refuses to run due to incorrect squid.conf
b) shorewall refuses to run due to incorrect /etc/shorewall/interfaces

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.


Reproducible: 

Steps to Reproduce:
Comment 1 Bjarne Thomsen 2014-12-28 22:18:46 CET
Created attachment 5765 [details]
New internal squid.conf file

This is a file which is actually used as /etc/squid/squid.conf.
squid seems to work, but I am not an squid expert.
I have taken the current squid.conf.default and merged it with
lines from the old internel squid.conf file.

CC: (none) => bjarne.thomsen

Comment 2 Bjarne Thomsen 2014-12-28 23:59:51 CET
There is something wrong with the attached squid.conf. I now again get access denied when I try to access external http pages. There were only very minor changes as compared to the previous one which was working. I have to to do some further tests.
Comment 3 Bjarne Thomsen 2014-12-29 10:20:30 CET
Created attachment 5766 [details]
A minimalistic internal squid.conf based on Mageia5, only.

It was probably not a bright idea to take 3129 as the forward proxy for http.
This new squid.conf is based on the one in mga5, only, plus mynetwork and
http_port 3128 intercept
http_port 8080

squid is now running on my 2 mga4 gateways with this squid.conf.
Today, I am going to buy an USB-hub so that I can instal m5b1 on my fitpc2.

Attachment 5765 is obsolete: 0 => 1

Comment 4 AL13N 2014-12-29 13:49:30 CET
why is it intercept and not transparent?

CC: (none) => alien

Comment 5 AL13N 2014-12-29 13:51:50 CET
also, regarding shorwall, are both eth0 and eth1 in the local interfaces in a clean mga5 ? i thought mga5 had device renaming and they were called enp3s0 or something like it?
Comment 6 Bjarne Thomsen 2014-12-29 14:39:49 CET
The shorewall interfaces are from mga4. The squid.conf file is taken from a clean install of mga5 in VirtualBox. Yes, the interfaces are called enp2s0. I managed to get mga5beta1 installed on the fitpc2, but unfortunately the standard KDE desktop produces a black screen and the ssh server is not installed, so I cannot get into the box that way. I installed mga3 more that 2 years ago. I remember that I had a lot of problems with the graphics chip. The next time I managed to avoid the problem by making a net upgrade to mga4.

I found the "intercept" by using google on the error message from squid.
Apparently "transparent" is deprecated. https is intercepted and http
forwarded to 8080, if my understanding is correct.

 ERROR: No forward-proxy ports configured
Daz's bits and bobs:
I had the above issue after a squid upgrade and after changing from http_port 3128 transparent to http_port 3128 intercept.

Add âhttp_port 8080â³ line to squid.config to avoid this message, if you are not already using that port.

The changes in security require that a separate port be setup for forwarding proxy requests
http://sigtar.com/2014/02/11/error-no-forward-proxy-ports-configured/
Comment 7 Bjarne Thomsen 2014-12-29 17:08:03 CET
I have not yet been able to get the graphics up and running, but I could login and run drakgw manually. cat /etc/shorewall/interfaces
#ZONE           INTERFACE               OPTIONS
net     enp2s0  detect
net     wlp0s29f7u7     detect
net     ens2    detect
net     ens3    detect
net     ens1    detect
loc     ens1    detect
loc     ens2    detect
loc     ens3    detect
loc     wlp0s29f7u7     detect

enp2s0 is connected to the "internet".
ens1 is connected to the local network.

ens2 and ens3 does not appear on the outside of the box.

wlp0s29f7u7 is WiFi.

All devices are on both loc and net.

After running drakfirewall:
#ZONE           INTERFACE               OPTIONS
net     enp2s0  detect
net     ens3    detect
net     ens2    detect
net     wlp0s29f7u7     detect
loc     ens1    detect
Comment 8 AL13N 2014-12-30 12:20:32 CET
how did the interfaces file look before running drakgw?

did you install only the internet interface during install? or the others as well?
Comment 9 AL13N 2014-12-30 12:38:58 CET
Created attachment 5772 [details]
patch to fix #14904

plz try to patch drakgw with this patch en retry to execute it. see if the interfaces are not anymore both in different zones...
Comment 10 AL13N 2014-12-30 12:39:51 CET
i replaced in the code transparent with intercept... but that's not enough, is it?

what else do you need to change after running drakgw?
Comment 11 Bjarne Thomsen 2014-12-30 13:20:44 CET
I installed the internet interface during install, only. I forgot to look at the interface file. Then I installed drakgw, and ran it through the internet interface. This required the installation of squid and bind.
I really hope that I can get the graphics running in fremebuffer mode. The old modesetting xorg.conf did not work.

Could I possible test your patch on drakgw in mga4 on the other gw-machine? I have already copied the mga5 version of drakgw to mga4.

No, intercept is not sufficient. You would have to enter my squid.conf into the perl code. 2 ports are really needed now. The internel squid.conf is really made for a very old version of squid.
Comment 12 Bjarne Thomsen 2014-12-30 14:19:34 CET
To me it looks as if a closing } is missing in line 177:
176               for my $loc_if (@{$shorewall->{loc_zone}}) {
177                   $shorewall->{net_zone} = [ grep {!/^$loc_if$/} @{$shorewall->{net_zone}} ];
Comment 13 Bjarne Thomsen 2014-12-30 16:09:12 CET
There is something completely wrong with my install on fitpc2.
The interface names are
enp2s0
ens1
ens2
ens3
I cannot bring ens1 up. Why are there 3 ens-interfaces?
Hardware detections tells med that there are 2 Realtek connections.
PCI domain       0          0
Bus PCI#         3          2
module r8169

The names should have been enp2s0 and enp3s0 or ?
There seems to have been a misidentification.
I also failed to obtain a usable screen in framebuffer mode.
My fitpc2 is useless for further testing of mga5.
Comment 14 Bjarne Thomsen 2014-12-30 17:45:09 CET
Here is the content of interfaces on windbox running mga4 before any changes:
net     wlan0   detect
net     eth0    detect
loc     eth1    detect

I then patched drakgw in mga5 with your changes (plus a closing "}")
and copied it to /usr/libexec/drakgw.

After disabling the local net by drakgw I obtain:
net     wlan0   detect
net     eth1    detect
net     eth0    detect

Next I share a local network by running drakgw once more:
net     wlan0   detect
net     eth0    detect
loc     eth1    detect

Finally I reboot windbox and obtain again:
net     wlan0   detect
net     eth0    detect
loc     eth1    detect

drakgw is working as expected in mga4.
I cannot see any reason that it should not work in mga5.

In this test I used my new version of squid.conf
Comment 15 Bjarne Thomsen 2014-12-30 18:52:41 CET
It turns out that dhcpd failed to start during the drakgw sequence.
It has to be started manually afterwards.
Comment 16 AL13N 2014-12-30 19:23:43 CET
the dhcp, is that on mga4 or mga5? did that give any errors, btw?

the missing } is indeed correct.

i'm glad my code worked...

can you check logs if it was tried to start at least?

systemctl status dhcpd.service (after drakgw, before start) should maybe give some useful info.

regarding squid, what do you mean a second port?
Comment 17 AL13N 2014-12-30 19:27:17 CET
regarding the device renaming, it could be shared ports... the correct one isn't necessarily the first...

you should check which has a link detected with ethtool... you may have to set the interface on, to detect a possible link, with the following command: "ip link set ensX up"

detection is as easy as "ethtool ensX"
Comment 18 Mageia Robot 2014-12-30 19:28:59 CET
commit 08d07c92584a7cf5da9762fd585fff8090a89def
Author: Maarten Vanraes <alien@...>
Date:   Tue Dec 30 19:27:51 2014 +0100

    make sure net and loc zones don't have the same interface (mga#14904)
---
 Commit Link:
   http://gitweb.mageia.org/software/drakx-net/commit/?id=08d07c92584a7cf5da9762fd585fff8090a89def
Comment 19 AL13N 2014-12-30 19:43:45 CET
btw: could you check if it's acceptible to have the second port (forwarding) identical as the intercept one?

ie:


http_port 3128
http_port 3128 intercept


technically it should be possible, but i donno if the restrictions do this?

if the above fails (try maybe also reverse order) try again with only:

http_port 3128
Comment 20 Bjarne Thomsen 2014-12-30 20:05:44 CET
From now all is in mga4. There were far too many mga5 bugs on the other box.
The actual kernel is 3.17.2. Isn't cauldron already at 3.18.2?
It turns out that the failure of dhcpd happened during a reboot.
dhcpd was running after the network sharing, but eth1 did not come up due to the problem with the squid configuration. So it looks as if the problem is the boot sequence of services.

I can try the different port options, but I think that the idea is to split
https and http trafic.

I can also try ethtool.
Comment 21 AL13N 2014-12-30 20:20:55 CET
no, the https and http are not related to the forward ports of squid.


the split is that the one port is used only for interception and the other is for the possible error pages of squid, or FTP directory entries...

i want to know both functions are possible on both ports, like it happened in the past with transparent...

i'm waiting on your tests on squid to commit this part...
Comment 22 Bjarne Thomsen 2014-12-30 21:38:48 CET
Interesting.

With
http_port 3128
alone, I can only access secure https pages,
otherwise I am told to contact the cache administrator.

With
http_port 3128 intercept
alone, there are lots of
ERROR: No forward-proxy ports configured.
in /var/log/squid/cache.log

But apparently it works with

http_port 3128 intercept
http_port 3128

I can access both http and https pages from machines
on the local net, and there are no ERROR lines in cache.log

So you are right. It is possible.
Comment 23 AL13N 2014-12-30 22:30:24 CET
thanks! i'll commit!
Comment 24 AL13N 2014-12-30 22:33:04 CET
ok, so i committed, but i didn't release drakx-net... i don't know how, can anyone do this so mga5 is fixed?

CC: (none) => mageia, pterjan, thierry.vignaud
Source RPM: drakx-net-text et al. => drakx-net

AL13N 2014-12-30 22:33:20 CET

CC: (none) => mageia

Comment 25 Mageia Robot 2014-12-30 22:34:30 CET
commit 156d1361a8584e743ad0626cedc28155fc78e54a
Author: Maarten Vanraes <alien@...>
Date:   Tue Dec 30 22:29:25 2014 +0100

    add a forward port too (mga#14904)
    
    apparently the same port can be used for interception and forwarding (as
    was the transparent functionality before)
---
 Commit Link:
   http://gitweb.mageia.org/software/drakx-net/commit/?id=156d1361a8584e743ad0626cedc28155fc78e54a
Comment 26 Bjarne Thomsen 2014-12-30 22:36:41 CET
This order
http_port 3128
http_port 3128 intercept

does NOT work!
Comment 27 Bjarne Thomsen 2014-12-31 06:28:09 CET
After
http_port 3128 intercept
http_port 3128
in the original squid.conf "squid -k pars" produces:
Processing: acl manager proto cache_object
2014/12/31 06:21:17| aclParseAclLine: ACL 'manager' already exists with different type.
FATAL: Bungled /etc/squid/squid.conf line 16: acl manager proto cache_object
Squid Cache (Version 3.4.10): Terminated abnormally.
After removing that line "squid -k pars" produces:
Processing: acl localhost src 127.0.0.0/8
2014/12/31 06:23:38| WARNING: (B) '127.0.0.1' is a subnetwork of (A) '127.0.0.0/8'
2014/12/31 06:23:38| WARNING: because of this '127.0.0.0/8' is ignored to keep splay tree searching predictable
2014/12/31 06:23:38| WARNING: You should probably remove '127.0.0.1' from the ACL named 'localhost'
2014/12/31 06:23:38| WARNING: (B) '127.0.0.1' is a subnetwork of (A) '127.0.0.0/8'
2014/12/31 06:23:38| WARNING: because of this '127.0.0.0/8' is ignored to keep splay tree searching predictable
2014/12/31 06:23:38| WARNING: You should probably remove '127.0.0.1' from the ACL named 'localhost'
2014/12/31 06:23:38| Processing: acl to_localhost dst 127.0.0.0/8
2014/12/31 06:23:38| WARNING: (B) '127.0.0.0/8' is a subnetwork of (A) '127.0.0.0/8'
2014/12/31 06:23:38| WARNING: because of this '127.0.0.0/8' is ignored to keep splay tree searching predictable
2014/12/31 06:23:38| WARNING: You should probably remove '127.0.0.0/8' from the ACL named 'to_localhost'

I now removed the 2 lines:
acl localhost src 127.0.0.0/8
acl to_localhost dst 127.0.0.0/8
After that "squid -k pars" produces NO WARNINGS.
Comment 28 Bjarne Thomsen 2014-12-31 06:48:12 CET
The local network is now shared on interface ans1 in mga5 on my fitpc2.
When will the patched drakx-net be available on cauldron, so I can test it
in mga5?
Comment 29 AL13N 2014-12-31 09:25:52 CET
in squid.conf did you also remove?

http_access allow manager localhost
Comment 30 Mageia Robot 2014-12-31 09:28:54 CET
commit a4919e1ec70bb0ec780260103674d8ad31243b0a
Author: Maarten Vanraes <alien@...>
Date:   Wed Dec 31 09:27:43 2014 +0100

    remove double definition and warnings (mga#14904)
---
 Commit Link:
   http://gitweb.mageia.org/software/drakx-net/commit/?id=a4919e1ec70bb0ec780260103674d8ad31243b0a
Comment 31 AL13N 2014-12-31 09:29:43 CET
(In reply to Bjarne Thomsen from comment #28)
> The local network is now shared on interface ans1 in mga5 on my fitpc2.
> When will the patched drakx-net be available on cauldron, so I can test it
> in mga5?

I committed in code, but i can't do a release of drakx-net... so, someone still needs to do this so cauldron can be retested...
Comment 32 Bjarne Thomsen 2014-12-31 10:55:18 CET
No. I have (in mga5):
http_access allow manager localhost
http_access deny manager
I do not really understand this.

The recommended minimum configuration in squid.conf.default has
# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager
so I kept this combination in my squid.conf in mga4.
Has the order a special meaning?
This was the reason that I created my own squid.conf
around the DEFAULT configuration.

Also in the OLD configuration
http_access deny to_localhost
should probably be removed, as it is no longer defined.
Also, it is not in the DEFAULT configuration.

Also, there are these in the OLD configuration
http_reply_access allow all
icp_access allow all
What do they do? They are NOT in the DEFAULT configuration,
and I do not see any change in functionality. What is icp
and reply_access? Are they a security problem?

And there are also these
memory_pools off
ie_refresh on
Why are they not in the DEFAULT configuration?
Comment 33 AL13N 2014-12-31 11:06:42 CET
i don't know, i guess that's something for the squid maintainer to see...

i suspect the localhost and to_localhost definitions are somewhere else, or it would've given some kind of warning...

anyway, right now i want to have something that works... the squid maintainer can look over the configuration and fix it better if it's to be done...

CC: (none) => cjw, ennael1, luigiwalser

AL13N 2014-12-31 11:07:17 CET

CC: (none) => luis.daniel.lucio

Comment 34 AL13N 2014-12-31 11:15:05 CET
ok, so i built a new release and asked for freeze pushes...
Comment 35 AL13N 2014-12-31 11:43:30 CET
and it appears i could submit myself... so, get drakx-net-2.14-1.mga5 and test it out, ok? if this is fixed, i'll put this on mga4 too...
Comment 36 Bjarne Thomsen 2014-12-31 13:26:59 CET
In the mean time I copied the patched drakgw to mga5.
After a sharing the internet the intervaces are now:
net     enp2s0  detect
loc     ens1    detect
loc     ens2    detect
loc     ens3    detect
loc     wlp0s29f7u7     detect

Is this how it should be?
I then re-booted the PC. squid was running, but shorewall was enabled
but inactive. According to systemctl status shorewall, it had not been
started, so I had to start it manually. dhcpd was running.
This is probably unrelated to drakgw.

It is a good idea to have the maintainer of squid look at squid.conf
Comment 37 David Walser 2014-12-31 13:35:01 CET
Some of the old localhost lines are built into newer versions of Squid and it now considers it invalid to have them explicitly written into squid.conf.  As long as you remove any invalid lines and squid runs without errors and does what it's supposed to do, then you're fine.  You shouldn't need the squid maintainer to figure that out.

CC: luigiwalser => (none)

Comment 38 Bjarne Thomsen 2014-12-31 14:02:31 CET
So, squid.conf is fine as long as "squid -k pars" does not complain?
All relevant services are now running after a "Sharing the internet..".
I have update to the new drakx-net packages.
The only problem is that shorewall does not appear to have been
started after a re-boot. I am not sure if this is related to
the changes in drakx-net?
Comment 39 Bjarne Thomsen 2015-01-20 00:04:12 CET
I am sorry, I am back again. This time with a shorewall problem.
I made a fresh install from m5b2 and reconfigured a LAN with drakgw.
After a re-boot I discovered that shorewall was dead and mandi was running.
It used to the other way around: shorewall was running and mandi was dead.
I am hoping that I can get some help figuring out why.

systemctl status shorewall.service:
â shorewall.service - Shorewall IPv4 firewall
   Loaded: loaded (/usr/lib/systemd/system/shorewall.service; enabled)
   Active: inactive (dead)

If I enter the "Set up your personal firewall" and select the interface
to protect, shorewall is running afterwards.

systemctl status shorewall.service:
â shorewall.service - Shorewall IPv4 firewall
   Loaded: loaded (/usr/lib/systemd/system/shorewall.service; enabled)
   Active: active (exited) since Mon 2015-01-19 23:33:49 CET; 57s ago
  Process: 8412 ExecStart=/sbin/shorewall $OPTIONS start (code=exited, status=0/SUCCESS)
 Main PID: 8412 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/shorewall.service

shorewall6 was running after the re-boot.
Comment 40 Bjarne Thomsen 2015-01-20 03:31:12 CET
It must be a timing problem between shorewall and mandi.
I disabled mandi. After a re-boot both shorewall and shorewall6 was running.
mandi is not a systemd service. How is the timing between the old-time
services and systemd services?
Comment 41 Bjarne Thomsen 2015-01-20 10:28:28 CET
This problem has absolutely nothing to do with the configuration of a LAN.
After a fresh install of m5b2 on another PC the same thing happens:
After a re-boot either mandi runs and shorewall is dead or shorewall
runs and mandi is dead.
I am moving further reports to Bug 14930.
Comment 42 Bjarne Thomsen 2015-01-23 06:13:41 CET
(In reply to Bjarne Thomsen from comment #22)
> Interesting.
> 
> With
> http_port 3128
> alone, I can only access secure https pages,
> otherwise I am told to contact the cache administrator.
> 
> With
> http_port 3128 intercept
> alone, there are lots of
> ERROR: No forward-proxy ports configured.
> in /var/log/squid/cache.log
> 
> But apparently it works with
> 
> http_port 3128 intercept
> http_port 3128
> 
> I can access both http and https pages from machines
> on the local net, and there are no ERROR lines in cache.log
> 
> So you are right. It is possible.

This is NOT true in mga4, or I missed the error message in the cache.log.
In mga4 the "Validation Procedure" considers it an ERROR.
#################################################################
http_port 3128 intercept
http_port 3128
#################################################################
2015/01/23 05:35:16 kid1| Accepting NAT intercepted HTTP Socket connections at
local=0.0.0.0:3128 remote=[::] FD 17 flags=41
2015/01/23 05:35:16 kid1| Accepting HTTP Socket connections at local=[::]:3128
remote=[::] FD 18 flags=9
....
2015/01/23 05:35:16 kid1| Beginning Validation Procedure
2015/01/23 05:35:16 kid1| ERROR: listen( FD 18, [::] [ job2], 256): (98)
Address already in use
2015/01/23 05:35:16 kid1|   Completed Validation Procedure

#################################################################
http_port 3128 intercept
http_port 8080
#################################################################
2015/01/23 05:39:03 kid1| Accepting NAT intercepted HTTP Socket connections at
local=0.0.0.0:3128 remote=[::] FD 17 flags=41
2015/01/23 05:39:03 kid1| Accepting HTTP Socket connections at local=[::]:8080
remote=[::] FD 18 flags=9
......
2015/01/23 05:39:03 kid1| Beginning Validation Procedure
2015/01/23 05:39:03 kid1|   Completed Validation Procedure

I hesitate to use a construction that the software itself
considers an ERROR.
Comment 43 Bjarne Thomsen 2015-01-23 14:09:15 CET
Indeed, I had overlooked this ERROR in cauldron/mga5.
In both cases it is considered an error to use the same port number.
Comment 44 Bjarne Thomsen 2015-02-16 20:26:20 CET
There is a syncronization problem between startup of old-style services
and systemd services due to hot-plogging of interfaces.
But this has nothing to do with the title of this report, so I consider
the bug as solved.

Status: NEW => RESOLVED
Resolution: (none) => FIXED


Note You need to log in before you can comment on or make changes to this bug.