Don't miss out on new posts! Sign up! Also, visit the VHT Forums!
vmbr0 without physi...
 
Notifications
Clear all

vmbr0 without physical uplink

17 Posts
2 Users
2 Likes
408 Views
(@life-from-scratch)
Posts: 14
Eminent Member
Topic starter
 

Hello! I watched your recent video on Proxmox VLANS and have a question that's sort of related to that. Currently I have a PfSense VM for my network router. When I first set it up, I assumed that Proxmox could get LAN connectivity over the vmbr0 interface. So in PfSense I set up a bridge between the virtual interface and LAN, but that didn't seem to work as I couldn't access the Proxmox webGUI. Next I plugged a physical cable from my LAN network to the motherboard NIC on my Proxmox box and that froze the entire network. So I deleted the bridge in PfSense between the virtual interface and LAN, plugged the physical cable back in and everything worked. But I'd still like to revisit that issue and eliminate the physical cable that uplinks vmbr0 to my LAN. 

I came up with a drawing to illustrate. The red cable is the one that I'd like to eliminate. Is this possible? Thanks!

Untitled Diagram

 

 
Posted : 16/12/2023 1:21 pm
Brandon Lee
(@brandon-lee)
Posts: 446
Member Admin
 

@life-from-scratch, thank you for signing up for the forums! Glad to help through this one. Just curious, can you post some screenshots of the virtual interface configuration and if any VLANs are in play? Also, just a reminder to post some pics and specs in the thread here (just reply to that thread) for a chance to win a mini PC. 👍 

 
Posted : 16/12/2023 1:34 pm
(@life-from-scratch)
Posts: 14
Eminent Member
Topic starter
 

Currently everything is pretty much default. I'm assuming that bottom interface that PfSense shows is the vmbr0 from Proxmox, the other four are on my PCI card. Proxmox network settings are absolutely stock aside from the static IP I gave it. eno1 is the motherboard nic. 

Screenshot (22)
Screenshot (23)
This post was modified 3 months ago by Life from Scratch
 
Posted : 16/12/2023 7:12 pm
Brandon Lee
(@brandon-lee)
Posts: 446
Member Admin
 

@life-from-scratch, one thing you can do to begin with is for sure identify which network adapter is being used in pfSense for the connection to vmbr0 in Proxmox. You can take a look at your NIC assigned in pfSense and see which mac address lines up with what you are seeing on the network adapter in Proxmox. Let me know what you find there.

image

 

 
Posted : 16/12/2023 7:45 pm
(@life-from-scratch)
Posts: 14
Eminent Member
Topic starter
 

Yes, the MAC address shown in Proxmox matches that last interface shown in PfSense. 

 
Posted : 16/12/2023 8:17 pm
Brandon Lee
(@brandon-lee)
Posts: 446
Member Admin
 

@life-from-scratch. It sounds like you just have a flat single VLAN network. Is that true? I think this should be pretty easy to configure, but it looks like where things are getting complicated is plugging the devices directly into your Proxmox host. What you need is a small network switch to aggregate your devices as things will probably be difficult to make work as expected plugging into the host directly. I know, especially with the small space for your lab, you are probably wanting to have as few cables as possible, but I think you will always be fighting unexpected behaviors trying to set it up this way. The Proxmox bridge isn't really designed to be a network switch in itself for physical resources. I think you would like to have a network switch with VLAN capabilities in the future as well if you want to delve into carving up your traffic even further for different traffic types, etc.

Ideally, to simplify, you could just VLAN-aware enable the bridge, and have all 4 connections backing the bridge. Then you can plug your devices into a VLAN aware switch and plumb traffic as you need from there. Let me know if you would like to continue to experiment though without a switch. I could probably set up a test host and play around with configurations from that angle.

 
Posted : 16/12/2023 9:01 pm
(@life-from-scratch)
Posts: 14
Eminent Member
Topic starter
 

Yes, no VLANs yet. I have a small switch already. I've messed with wiring don't remember exactly what's connected where at the moment (see cable management lol). It all works fine, I just literally have a cable with one end plugged into the network card that's passed through to PfSense and the other end plugged into the motherboard of the same machine. It just seems like since PfSense and all my VM's are connected to the virtual bridge that that physical cable shouldn't be needed. I would still have the PfSense LAN port going out to the switch.

 
Posted : 16/12/2023 9:15 pm
Brandon Lee
(@brandon-lee)
Posts: 446
Member Admin
 

Ah ok gotcha. Passthrough will change things a bit. With passthrough, that VM will have exclusive access to that physical device. So it makes sense that you would need to somehow tie this into your vmbr0 bridge. Without passthrough, you could have pfSense and Proxmox be able to communicate "in the box." Here is a way you could set that up.

With a few VLANs, you could carve up that traffic so that everything is switched virtually inside Proxmox between pfSense, your Proxmox host, and everything else. 

As an example, you could create:

VLAN 100 - WAN 

VLAN 200 - LAN

VLAN 300 - other traffic if needed

That way, you could plumb everything in on the pfSense side virtually. You could have a virtual interface in pfSense tagged with VLAN 100 for WAN, a second virtual interface tagged with VLAN 200 for LAN traffic, and an optional interface tagged with VLAN 300. With this configuration, you could have a single cable out of your Proxmox host into your switch on a trunk port, and then have your devices plugged into access ports tagged with the appropriate VLANs if that makes sense.

 
Posted : 16/12/2023 9:33 pm
(@life-from-scratch)
Posts: 14
Eminent Member
Topic starter
 

I think I understand, but not really. Why did it not work to simply set up a bridge in PfSense between the virtual interface from Proxmox and LAN? In PfSense I can make one of my physical ports into WAN and set the remaining three physical ports on that card to be LAN right? Then the extra ports are just behaving like a switch, correct? Why doesn't it work to just include the virtual interface in that bridge?

 
Posted : 16/12/2023 9:58 pm
(@life-from-scratch)
Posts: 14
Eminent Member
Topic starter
 

I think I'm getting what you're talking about with not passing through. I'm not trying to push the whole of my LAN through virtually. I'm only talking about providing LAN connectivity to my VM's and Proxmox GUI through the virtual interface. 

 
Posted : 16/12/2023 10:09 pm
Brandon Lee
(@brandon-lee)
Posts: 446
Member Admin
 

Hmm, yeah, it sounds like it has created a bridging loop, which amounts to a broadcast storm, and that is the behavior you saw with it shutting down your network, it sounds like. It may not work since you are combining the two bridges for the one VM if I am thinking about this correctly and that is what causes a bridge loop if there are more than one layer 2 path between devices. So, it may not work to try to include a virtual interface from the Linux bridge.

 
Posted : 16/12/2023 10:16 pm
(@life-from-scratch)
Posts: 14
Eminent Member
Topic starter
 

I guess next time I feel like breaking things again I should try it again and see if I'm able to access my other services and if it's just the Proxmox webGUI that I can't reach. Because if I created a loop, that means the bridge was doing something and unplugging the physical cable should break the loop. 

 
Posted : 17/12/2023 10:54 am
Brandon Lee
(@brandon-lee)
Posts: 446
Member Admin
 

@life-from-scratch it would be interesting to see what happens with more testing and see if you are still unable to get to the web UI on the same setup or if this was due to something else at the time. Keep me posted on what you find there 👍 

 
Posted : 17/12/2023 10:59 am
(@life-from-scratch)
Posts: 14
Eminent Member
Topic starter
 

Well.... I unplugged the cable from my switch back to the Proxmox node and inside PfSense just bridged the LAN and virtual interface together and for some reason this time it just seems to work. All my services seem to be up and I can access the Proxmox dashboard. So I'm not sure why it didn't work when I initially set it up, but it does now.

This opens up a lot more options for machines to run a "forbidden router" setup on like Hardware Haven did. I get why bare metal routers are preferred, but I also like the idea of being able to run other critical services (Home Assistant for me) on the same machine. Running like this it should work with only two nics. One for WAN, one for LAN and not needing a third to for the hypervisor. 

 
Posted : 17/12/2023 5:48 pm
Brandon Lee
(@brandon-lee)
Posts: 446
Member Admin
 

@life-from-scratch that is great! That is cool that it works as you expected it to the first time. Keep me posted on how it performs and operates moving forward.

 
Posted : 17/12/2023 8:17 pm
Page 1 / 2