Help  Search  Portal
 Portal  Search
Hello There, Guest!  Register  Login

Experimenting with network topology and throughput.

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
I am running a new router in my home now. A Celeron 3865U mini PC with 6 ports. I'm not sure if a Celeron is capable of switching gigabit speeds, so tested it out with the iPerf benchmark tool.

So what iperf is doing is really testing the maximum transfer speed (upload or download) of your network. You need a computer running iPerf in server mode (in my case the router), and then run clients at various computers to test the speed.

This is the results of running iPerf in one of the development machines.
Client connecting to 10.x.x.x, TCP port 5001
TCP window size:  325 KByte (default)
[  3] local 10.x.x.x port 42884 connected with 10.x.x.x port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.15 GBytes   987 Mbits/sec

987 Mbits, not too bad. Good to know I'm wrong. Celeron is capable of maxing out a gigabit network.

Next, I try this from my NAS.
[  3] local 10.x.x.x port 33607 connected with 10.x.x.x port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1014 MBytes   851 Mbits/sec

Not as quick, but still acceptable. Kind of expected because this NAS is running on HP N54L, and I believe it's using Marvel  NIC. This network card is slower than Intel (and consume more power).

And this is the throughput from my primary music player (running Snakeoil of course):
[  3] local 10.x.x.x port 58196 connected with 10.x.x.x port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   625 MBytes   524 Mbits/sec

524 Mbit/s. This is expected because I am using a PCI network card, PCI maxes out at 533 Mbit/s. So I'm already very close to the limit. Besides USB is 480 Mbits/s so this is OK.

Finally, I run this on the Raspberry Pi 3... And what an appaling result!!
[  3] local 10.x.x.x port 35116 connected with 10.x.x.x port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   338 MBytes   284 Mbits/sec

284 Mbits/s! Did I got that right?

A quick google couldn't tell me anything conclusive. But it does appear network is a bottle neck with these Pis. As good as they are, there are restrictions when it's a SoC.

USB 2.0 max out at 480 Mbits/s  anyway, so I reckon even at this speed the Pi should be able to handle high res or DSD relatively well. But there's not much room for error here.

ODroids, Banana Pis can sustain higher throughputs. They are also more expensive.

Anyway, why the new router? I am finally going to bite the bullet and seggerate everything in my network into it's own VLAN. Not entirely sure what that'd do, so watch this space.
Snakeoil Operating System - Music, your way!
So before we begin, let's introduce the hardware:

This is the new router:
Purchased off Aliexpress. It took about a month to deliver to Perth.

Here's the unit, plus:
  • 2x 8GB of 2133 MHz RAM puchased locally
  • Reused an old 500 GB HDD I have lying around somewhere

Here's a look of the insides. This is the base of the unit. Behind this board sits the CPU, and that's attached to the top of the case, where all the heat sink fins are. The picture doesn't give you a scale of how small and compact this unit is. The machining is done to millimeter perfection! There is no gap for me to pry the bottom cover off. I have to remove one of the sides so I can remove the top.

And here it is with RAM and HDD fitted. This should give you a scale of how compact this unit unit is.

Last but not least, my network is now broken up into:

So what is a VLAN? It stands for Virtual Local Area Network. In a network switch, all computers have the capability to talk to any other computer on this same switch. A virtual LAN is like inception, it creates another switch inside this switch (hence the word virtual), and computers can only talk to another computer if they are on the same virtual switch.

So in the above example, computers in "DEV" can only see other computers in "DEV". If there is a need to talk to computers outside the virtual switch, they need to go through the router.

In this setup, I have broken down the network communications logically, this in turn prevents a lot of unnecessary suprious network traffic on the network. The theory is, the overheads of VLAN will outweigh the constant traffic noise that is bombaring all the ports on the network. 

So if I design this right, the "Music" VLAN will be where my music server and players are. It can still communicate with any computer on the other VLAN, but it will no longer see the unnecessary broadcast messages (This is the noise I am referring to).

For this to work, the router has to be extrememly fast, and be capable of receiving and forwardinig packets at a moment's notice. Hence the choice of the Minisys router. It has 6 network ports. 1 is connected to the Internet, 4 goes back to the switch (This is called LAGG).

Don't worry if you don't understand any of the above. Big Grin

The unit is powered off a 12 V SMPS. Not ideal for sound quality, so it'd be something I'd look at addressing in the future.

For folks who are interested in specs, here's the router specifications:

There are faster versions - fanless i5 and even an i7 in a different case. But a router is designed to run 24x7. Figures a Celeron will be better for the environment.

My only worry was it is incapable of switching at the maximum speed. For now that seems to be OK. I doubt the Celeron can do full 6 ports at 1 gbps, but that is not a requirement for me.

My quick tests have showed me, not all network chips are built the same. Some chips are just faster than the others. The fastest chips so far are still made from Intel.

I don't  use any audiophile network cables in my setup at all. But in the future if I get a chance to try them out, will be interested to see the iPerf results Big Grin .. My guess is they'd be the same in terms of throughput.
Snakeoil Operating System - Music, your way!
Been busy lately so havn't had the time to sit down for a proper audition.. But did managed to sneak in a couple hours last night and on Sunday...

I havn't totally isolated the NAS with my CAP yet, but removed enough of the other machines off to their own VLANs and thought that should make a good difference.

So what's my first impressions? Not so good unfortuantely..

[Image: bad_atmosphere.gif]

Not entirely sure what is happening, just do not feel that engaged with the music with this new setup now. The music is boring and lifeless...

This is weird because technically the packets are transferred local to the switch, and not to the router. So everything should be the same as before - with the exception of less spurious network traffic and more VLANs..

More VLANs. Maybe I should stick to just 2 VLANs - one for music, and one for everything else, instead of trying to be too clever and seperate out everything. :shrug: D'oh.

Will continue to try different things over the next few days and see how it goes.
Snakeoil Operating System - Music, your way!
(06-Nov-2018, 10:08 AM) agent_kith Wrote: Not entirely sure what is happening, just do not feel that engaged with the music with this new setup now. The music is boring and lifeless...

Man, I should really keep this thread up to date! It's been > a year since the last update. For historical reasons, the solution to fix the above "boring" sound is to ground the case of this tiny PC. So all I did is to attach a thin lean wire. Tap one end to this router, and the other end to my development machine (which is grounded). Problem solved.

Second tweak that works is for me to start using Jumbo Frames. See this  blog article when I turned off Jumbo Frames and found the music was perceptally worse. Long story short the network card I was using on my NAS is causing random reboots. This isn't completely fixed yet, I'll need to buy a new NIC to fix this permanently.

Anyhoo the routing software running on this tiny PC is something called pfSense. Over the year I have experimented with various ways of connecting things up (see here for a quick review). And right now I've finalised on the L3 topology. For a long time this setup is running fine. Then I stupidly tried to change a setting on the router. I can't remember what that was but it brought down the whole network. Big Grin Luckily pfSense has this great feature - undo configuration change. So all I need to do is to roll-back to a previous configuration.
[Image: pf-Sense-restore-config.png]

Or so I thought! Everything seems to work, but for some stupid reason Jumbo frames no longer cross VLANs (sometimes it works, and sometimes it doesn't).... Honestly NFI what the problem could be. I am pretty sure if I factory reset the settings back to default, and re-apply the configuration everything will work as before. But I really shouldn't need to do that! All these little problems I constantly get with pfSense, this is the last straw!

So come next year, I'm gonna ditch pfSense and go with Untangle. Good thing is all I need to do is to swap out the HDD to install Untangle. And I can always swap the old pfSense HDD back to get the old system back.

So many things to do, so little time [Image: shock1.gif] .
Snakeoil Operating System - Music, your way!
Quote:So many things to do, so little time
+1 !

Hope you have a wonderful and relaxing break in Honkers and Taiwan this festive season... you deserve it !!
[-] The following 1 user Likes Bromf's post:
  • agent_kith
Here's a second experiment. What happens if I want to send something with a payload that's greater than 9000 bytes, but the receiving party cannot accept this number? For example with Raspberry Pis, the maximum allowable MTU is 1500.

Now the router will be smart enough to break it up into smaller packets and send it to the receiving party. The split up packets will be marked so that it can be re-assembled into it's original packets.

Like so, I am trying to send a payload of 9001 bytes to a machine with a MTU of 9000. If I have the "do not fragment" bit set, this original 9001 bytes cannot be split up and must be sent as is, or be rejected. But if I don't have this bit set, the payload can be split into 2. In Microsoft Windows, the "do not fragment" flag is -f. So with this on, trying to send 9001 bytes will fail, and succeed if off. Exactly as designed when I'm using Untangle as the router:
[Image: Jumbo-Routing-Fragment.png]

Frustratingly, I ran the same experiments before in pfSense, and it worked! Now this behaviour is intermittent and I have ny idea why. One thing that bugs me most is the use of "double negatives" in pfSense configuration. Turning something on (tick in a checkbox) means I'm actually turning something off!

There are features in pfSense that I will miss (HAProxy and ACME). But thanks to the generous support by you guys, I now have a Ryzen server with more grunt and RAM. I can now now create a new virtual machine to run HAProxy and ACME. In fact, I can even run the Untangle firewall as a virtual machine, keeping things even simpler!

I don't expect a lot of people to push their music setup to this extent. But if you are interested, remember to stay tune to the blogs to find out how I do it.. It'll also serve as a written record for me to refer back when things are broken again.
Snakeoil Operating System - Music, your way!
SnakeoilOS Mission

Our mission is to create a free to use computer OS that is easy to install, intuitive to operate and play music that will connect and engage with you emotionally.

SnakeoilOS gives you the freedom to spend more time on listening, enjoying and exploring music. Wasting time on computers is now a thing of the past! Everything is constantly evolving/improving. Please check back often for updates.

If you like this project, do show your support with a small token donation. All donations collected will be used to run this website, and for purchasing new equipment for the project.

Latest Threads

Users browsing this thread: 1 Guest(s)