[pfSense Support] Splitting a /24 into multiple subnets

[Andreas Kaiser]

Hi all,


first: I'm not really a network guy, but thanks to pfSense was able
to some advanced (at least by my measures) stuff by myself - until
now... So please be patient with me.

A Vmware host machine has 1 NIC and uses 1 public IP itself.
A second public IP (say 4.3.2.17/32) is used for the pfSense VM's
WAN interface. The provider is routing a /24 (say 1.2.3.0/24) on
that second IP.

If I configure pfSense's LAN to 1.2.3.1/24 everything works as
expected.

Now I'm trying to segment the /24 into 4 subnets with the pfSense
interfaces being:

-   1.2.3.1/26
LAN, connected to Vmware vSwitch1
used as the VMs' primary IPs

-   1.2.3.129/25
OPT1, connected to Vmware vSwitch2
to be used for SSL sites

The remaining segments shall be used later for various VPNs
(1.2.3.64/27, 1.2.3.96/28, 1.2.3.112/28).

Several Linux webserver VMs have 2 NICs each, connected to vSwitch1
on eth0 and vSwitch2 on eth1.

I've successfully configured pfSense to:

-   do everything related to 1.2.3.0/26 from the pfSense box
itself as well as from any host on the internet

-   being able to reach pfSense's 1.2.3.129/25 interface
from the pfSense box itself and from the internet

-   being able to reach the machines in the 1.2.3.128/25
from the pfSense box itself

I'm currently failing in reaching any of the VMs via their interfaces
connected to the 1.2.3.128/25. I've configured firewall rules to
allow ICMP echo requests as well as TCP ports 80 and 443 for
destinations in that subnet on the WAN interface. I can see that
traffic is blocked when I disable these rules and is passed if I
leave them enabled. If I do an HTTP request, I see
CLOSED:SYN_SENT/SYN_SENT:CLOSED in pfSense's "Diagnostics:
Show States".

If i do an HTTP request on an IP in the 1.2.3.0/26, everthing is
fine and I see "FIN_WAIT_2:FIN_WAIT_2" in the states table.

Any pointers (especially RTFMs with URLs or page numbers from "the
book") on what I'm missing are greatly appreciated.


TIA,

Andreas



-
To unsubscribe, e-mail: support-unsubscr...@pfsense.com
For additional commands, e-mail: support-h...@pfsense.com

Commercial support available - https://portal.pfsense.org

RE: [pfSense Support] Splitting a /24 into multiple subnets

[Tim Dickson]

> Now I'm trying to segment the /24 into 4 subnets with the pfSense interfaces 
> being:

It sounds easy enough - but may be because I'm not understanding exactly what 
you want.
But the simplest method I could come up with would be to setup your WAN to 
accept every IP your ISP routes to you, then do 1:1 to each internal network 
you need.
Create each internal network on a separate interface (either physical or VLAN)
Then set the RULES inbound on your WAN interface as needed.
That allows you to do any routing you want between interfaces / WAN and gives 
you granular control of everything.

-Tim


-
To unsubscribe, e-mail: support-unsubscr...@pfsense.com
For additional commands, e-mail: support-h...@pfsense.com

Commercial support available - https://portal.pfsense.org

Re: [pfSense Support] Splitting a /24 into multiple subnets

[Andreas Kaiser]

Am 23.05.2011 um 23:25 schrieb Tim Dickson:

>> Now I'm trying to segment the /24 into 4 subnets with the pfSense interfaces 
>> being:
> 
> It sounds easy enough

Maybe for you… ;-)

> - but may be because I'm not understanding exactly what you want.
> But the simplest method I could come up with would be to setup your WAN to 
> accept every IP your ISP routes to you, then do 1:1 to each internal network 
> you need.

Does that mean configuring

  1. a virtual IP of type "Proxy ARP" on the WAN interface for "IP Address(es)" 
of type "Network" with value "1.2.3.0/24" under  "Firewall: Virtual IP Address: 
Edit"

  2. one NAT 1:1 entry for each of the desired subnets under "Firewall: NAT: 
1:1: Edit", i.e.
 
 - external: 1.2.3.1, internal 1.2.3.1/26, NAT reflection disable

 - external: 1.2.3.129, internal 1.2.3.129/26, NAT reflection disable

?

> Create each internal network on a separate interface (either physical or VLAN)

I did that already.

> Then set the RULES inbound on your WAN interface as needed.

Would I still be able to filter traffic originating from LAN/OPT1 on their 
respective firewall ruleset?

> That allows you to do any routing you want between interfaces / WAN and gives 
> you granular control of everything.

*That* is exactly what I want ;-)


Thanks a lot,

Andreas
-
To unsubscribe, e-mail: support-unsubscr...@pfsense.com
For additional commands, e-mail: support-h...@pfsense.com

Commercial support available - https://portal.pfsense.org

Re: [pfSense Support] Splitting a /24 into multiple subnets

[David Burgess]

On Mon, May 23, 2011 at 4:14 PM, Andreas Kaiser  wrote:

>> That allows you to do any routing you want between interfaces / WAN and 
>> gives you granular control of everything.
>
> *That* is exactly what I want ;-)

Have you turned off automatic outbound NAT and disabled or deleted all
the automatically created rules for every interface that has a part of
the /24 public subnet?

db

-
To unsubscribe, e-mail: support-unsubscr...@pfsense.com
For additional commands, e-mail: support-h...@pfsense.com

Commercial support available - https://portal.pfsense.org

Re: [pfSense Support] Splitting a /24 into multiple subnets

[Eugen Leitl]

On Mon, May 23, 2011 at 02:25:26PM -0700, Tim Dickson wrote:
> > Now I'm trying to segment the /24 into 4 subnets with the pfSense 
> > interfaces being:
> 
> It sounds easy enough - but may be because I'm not understanding exactly what 
> you want.
> But the simplest method I could come up with would be to setup your WAN to 
> accept every 
> IP your ISP routes to you, then do 1:1 to each internal network you need.

So you basically pick the largest subnet containing all your networks,
and put that on the WAN interface?

> Create each internal network on a separate interface (either physical or VLAN)
> Then set the RULES inbound on your WAN interface as needed.
> That allows you to do any routing you want between interfaces / WAN and gives 
> you granular control of everything.

-- 
Eugen* Leitl http://leitl.org";>leitl http://leitl.org
__
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE

-
To unsubscribe, e-mail: support-unsubscr...@pfsense.com
For additional commands, e-mail: support-h...@pfsense.com

Commercial support available - https://portal.pfsense.org

Re: [pfSense Support] Splitting a /24 into multiple subnets

[Andreas Kaiser]

Am 24.05.2011 um 00:45 schrieb David Burgess:

> On Mon, May 23, 2011 at 4:14 PM, Andreas Kaiser  
> wrote:
> 
>>> That allows you to do any routing you want between interfaces / WAN and 
>>> gives you granular control of everything.
>> 
>> *That* is exactly what I want ;-)
> 
> Have you turned off automatic outbound NAT

Yes. Outbound is set to "Manual" with no mappings defined.

> and disabled or deleted all the automatically created rules for every 
> interface that has a part of
> the /24 public subnet?

Yes. No autocreated rules except "RFC 1918 networks", "Reserved/not assigned by 
IANA" on WAN and "Anti-Lockout Rule" on LAN.
-
To unsubscribe, e-mail: support-unsubscr...@pfsense.com
For additional commands, e-mail: support-h...@pfsense.com

Commercial support available - https://portal.pfsense.org

Re: [pfSense Support] Splitting a /24 into multiple subnets

[Frank Heydlauf]

Hi Andreas,

On Mon, May 23, 2011 at 11:24:48PM +0200, Andreas Kaiser wrote:
..
> A Vmware host machine has 1 NIC and uses 1 public IP itself.
> A second public IP (say 4.3.2.17/32) is used for the pfSense VM's
> WAN interface. The provider is routing a /24 (say 1.2.3.0/24) on
> that second IP.


let's draw a chart (use monospaced font!):

ISP
 |
 |
 |
  4.3.2.17  
   WAN
  pfSense 
 NAT+Filter
 LAN   OPT1
  1.2.3.1/261.2.3.129/25  
  | |
  | |   <-- VMware virtual switch
  | |
  1.2.3.5   1.2.3.155  (for example)
eth0eth1
Webserver

ist this correct?

And the default-route of "Webserver" goes to 1.2.3.1 ?

Ist the /24 your provider assigned to you routed to 4.3.2.17
or is 4.3.2.17 part of that /24 ?


-- 
Gruss Frank

-
To unsubscribe, e-mail: support-unsubscr...@pfsense.com
For additional commands, e-mail: support-h...@pfsense.com

Commercial support available - https://portal.pfsense.org

Re: [pfSense Support] Splitting a /24 into multiple subnets

[Andreas Kaiser]

Hi Frank!

Am 24.05.2011 um 09:57 schrieb Frank Heydlauf:

> On Mon, May 23, 2011 at 11:24:48PM +0200, Andreas Kaiser wrote:
> ..
>> A Vmware host machine has 1 NIC and uses 1 public IP itself.
>> A second public IP (say 4.3.2.17/32) is used for the pfSense VM's
>> WAN interface. The provider is routing a /24 (say 1.2.3.0/24) on
>> that second IP.
> 
> 
> let's draw a chart (use monospaced font!):
> 
>ISP
> |
> |
> |
>  4.3.2.17  
>   WAN
>  pfSense 
> NAT+Filter
> LAN   OPT1
>  1.2.3.1/261.2.3.129/25  
>  | |
>  | |  <-- VMware virtual switch
>  | |
>  1.2.3.5   1.2.3.155  (for example)
>eth0eth1
>Webserver
> 
> ist this correct?

This is absolutely correct, except that I'm using separate switches (vSwitch1, 
vSwitch2) for LAN/eth0 and OPT1/eth1.

I'm not sure what you mean by "NAT+Filter" – there isn't any NAT related stuff 
configured at all.

> And the default-route of "Webserver" goes to 1.2.3.1 ?

Yes.

> Ist the /24 your provider assigned to you routed to 4.3.2.17 or is 4.3.2.17 
> part of that /24 ?

It's just routed to 4.3.2.17. To be more specific WAN is 4.3.2.17/27 with it's 
gateway being 4.3.2.1/27. This gateway is tho only one in "System: Gateways" 
and shown as "WAN (default)". No routes are shown in "System: Static Routes".


Thanks for your efforts – to all of you!

Andreas
-
To unsubscribe, e-mail: support-unsubscr...@pfsense.com
For additional commands, e-mail: support-h...@pfsense.com

Commercial support available - https://portal.pfsense.org

Re: [pfSense Support] Splitting a /24 into multiple subnets

[Steve Haavik]

I'm currently failing in reaching any of the VMs via their interfaces
connected to the 1.2.3.128/25. I've configured firewall rules to
allow ICMP echo requests as well as TCP ports 80 and 443 for
destinations in that subnet on the WAN interface. I can see that
traffic is blocked when I disable these rules and is passed if I
leave them enabled. If I do an HTTP request, I see
CLOSED:SYN_SENT/SYN_SENT:CLOSED in pfSense's "Diagnostics:
Show States".


I've done a few setups like this. Make sure you have rules allowing the 
return traffic from OPT to WAN. To make sure you aren't getting bit by 
your webserver virtual machine routing the return traffic out the other 
interface (I don't know, it could happen...) try to setup a vm that only 
connects to vswitch2 and see if you have the same problem. Can you ping 
the vm from the firewall? Try pinging from each interface on the firewall.


If you can ping it fine when it's only connected to one vswitch, but not 
when you add the second interface to the vm it's probably a routing issue 
on the vm. Do you have default routes set for both interfaces on the 
webserver? If you do "netstat -rn" you should see entries for both 
interfaces. Something like this:


Destination Gateway   Genmask   Flags   MSS Window  irtt Iface
0.0.0.0 1.2.3.1   0.0.0.0   UG0 0  0 eth0
0.0.0.0 1.2.3.129 0.0.0.0   UG0 0  0 eth1


-
To unsubscribe, e-mail: support-unsubscr...@pfsense.com
For additional commands, e-mail: support-h...@pfsense.com

Commercial support available - https://portal.pfsense.org

Re: [pfSense Support] Splitting a /24 into multiple subnets

[Frank Heydlauf]

Hi Andreas,

On Tue, May 24, 2011 at 10:40:41AM +0200, Andreas Kaiser wrote:
> Am 24.05.2011 um 09:57 schrieb Frank Heydlauf:
...
> > let's draw a chart (use monospaced font!):
> > 
> >ISP
> > |
> > |
> > |
> >  4.3.2.17  
> >   WAN
> >  pfSense 
> > NAT+Filter
> > LAN   OPT1
> >  1.2.3.1/261.2.3.129/25  
> >  | |
> >  | |<-- VMware virtual switch
> >  | |
> >  1.2.3.5   1.2.3.155  (for example)
> >eth0eth1
> >Webserver
> > 
...
> > And the default-route of "Webserver" goes to 1.2.3.1 ?
> 
> Yes.

If you ping 1.2.3.155 from outside (ISP), the answer packets
will return via eth0 and 1.2.3.1.
At this point you may (probably will) hit anti-spoofing rules
and stateful filter rules at LAN interface.

=> You'll have to separate your answer-traffic on your
web-server based on rules, i.e. source routing based
on tcp source-port 443  or 80
or doing it in a more general way:


There are other options with NAT, proxies etc - but IMO none
of them better.

-- 
Gruss Frank

-
To unsubscribe, e-mail: support-unsubscr...@pfsense.com
For additional commands, e-mail: support-h...@pfsense.com

Commercial support available - https://portal.pfsense.org

Re: [pfSense Support] Splitting a /24 into multiple subnets

[Andreas Kaiser]

Hi Steve,


Am 24.05.2011 um 13:20 schrieb Steve Haavik:

>> I'm currently failing in reaching any of the VMs via their interfaces 
>> connected to the 1.2.3.128/25. I've configured firewall rules to allow ICMP 
>> echo requests as well as TCP ports 80 and 443 for destinations in that 
>> subnet on the WAN interface. I can see that traffic is blocked when I 
>> disable these rules and is passed if I leave them enabled. If I do an HTTP 
>> request, I see CLOSED:SYN_SENT/SYN_SENT:CLOSED in pfSense's "Diagnostics: 
>> Show States".
> 
> I've done a few setups like this. Make sure you have rules allowing the 
> return traffic from OPT to WAN. To make sure you aren't getting bit by your 
> webserver virtual machine routing the return traffic out the other interface 
> (I don't know, it could happen...) try to setup a vm that only connects to 
> vswitch2 and see if you have the same problem. Can you ping the vm from the 
> firewall? Try pinging from each interface on the firewall.
> 
> If you can ping it fine when it's only connected to one vswitch, but not when 
> you add the second interface to the vm it's probably a routing issue on the 
> vm. Do you have default routes set for both interfaces on the webserver? If 
> you do "netstat -rn" you should see entries for both interfaces. Something 
> like this:
> 
> Destination Gateway   Genmask   Flags   MSS Window  irtt Iface
> 0.0.0.0 1.2.3.1   0.0.0.0   UG0 0  0 eth0
> 0.0.0.0 1.2.3.129 0.0.0.0   UG0 0  0 eth1

that was the missing hint in the right direction – problem solved!

Indeed the return traffic had been sent through eth0, even if it originally 
arrived through eth1. The solution was to create a custom routing table as 
described on Darien Kindlund's blog: 
http://kindlund.wordpress.com/2007/11/19/configuring-multiple-default-routes-in-linux/

root@test:~# echo "1 ssl_sites" >> /etc/iproute2/rt_tables 
root@test:~# ip route add 1.2.3.128/25 dev eth1 src 1.2.3.150 table ssl_sites
root@test:~# ip route add default via 1.2.3.129 dev eth1 table ssl_sites
root@test:~# ip rule add from 1.2.3.150/32 table ssl_sites
root@test:~# ip rule add to 1.2.3.150/32 table ssl_sites



Thank you all for your tremendous support!

Andreas
-
To unsubscribe, e-mail: support-unsubscr...@pfsense.com
For additional commands, e-mail: support-h...@pfsense.com

Commercial support available - https://portal.pfsense.org