Extending cpuset to include user processes
#1
Would like to see the ability for a user to add a process to be included in cpuset. E.g. ​​would like dedicate a processor for CamillaDSP
p/s tried to manualy add cpuset but it appears that cpusets are reset at boot
Reply

#2
Good idea. Really worth exploring CPUSet to see what it can do..

Havn't had much time to work on the new GUI past week, and unlikely to have the time with the coming long weekend. But will try and incorp this into the new GUI rework. Issue created and tracked here.
Snakeoil Operating System - Music, your way!
[-] The following 2 users Like agent_kith's post:
  • patricia1066, Snoopy8
Reply

#3
Some more information for next release of cpuset:
  1. cpuset does not stop IRQs from using the user cpuset (CPUs 2,3).  Ideally, to reduce latency, IRQs should be on a different CPU to the music player.  My workaround is to edit
    /etc/default/irqbalance and set IRQBALANCE_BANNED_CPUS=C  (hex mask value, run in CPUs 0,1 not in CPUs 2,3)
  2. This may help with setting a new cpuset for a user process.  CamillaDSP was started as a system process at boot. Manually shifted using
    sudo cset proc --move --toset=user --pid 723 --threads

    Confirmed move worked using cset shield --shield -v  
    Code:
    cset: "user" cpuset of CPUSPEC(2-3) with 11 tasks running
       USER       PID  PPID SPPr TASK NAME
       -------- ----- ----- ---- ---------
       root       723     1 Sf70 [camilladsp]
       root       738     1 Sf70 [camilladsp]
       root       746     1 Sf70 [camilladsp]
       root       747     1 Sf70 [AlsaPlayback]
       root       748     1 Sf70 [AlsaCapture]
       root      2125     1 Soth [mpd]
       root      2174     1 Soth [io]
       root      2175     1 Sf40 [rtio]
       root      2178     1 Soth [player]
       root      2182     1 Soth [decoder]
       root      2387     1 Sf40 [output]
    cset: done
  3. Some issues with existing cpuset, a few player processes were not shifted to the user cpuset
    LMS + Squeezlite R2 : squeeze2upnp-x8  (using DLNA/UPnP bridge) remained in the system cpuset
    LMS + upmpdcli : squeeze2upnp-x8, mpd  remained in the system cpuset
    Minimserver : uses Java which remained in the system cpuset
Can confirm SQ improved with shifting IRQs to system cpuset and CamillaDSP to user cpuset.

*****
Added 22-Jun

A few more notes
  1.  Can move a few more kernel processes from user to system cpuset using
    sudo cset shield -k on
    While I heard no discernible difference with SQ, for completeness, should include this.
  2. With upmpdcli,  it is not possible to shift the underlying mpd process from system to user cpuset.  Given that upmpdcli is only a wrapper and the work is done by mpd, cpuset may not provide a SQ benefit for upmpdcli?
  3. Tried dedicating a cpu to LMS and another cpu to Squeezelite R2.  No difference in SQ to running both LMS and Squeezelite R2 in same cpu.
  4. Similarly no difference in SQ with mpd and CamillaDSP running on same cpu or on separate individual cpus.
  5. However, with cpu intensive servers like Roon or HQplayer, there may be a benefit in running their players on a seperate cpu?  Someone should test this.
  6. If mympd is enabled (coming in next release), should shift it from system to user cpuset. I do not use mympd and will disable it.
[-] The following 2 users Like Snoopy8's post:
  • agent_kith, hkphantomgtr
Reply

#4
Have updated the ticket with last post. Hoping to incorp this into the new GUI in the upcoming release.
Snakeoil Operating System - Music, your way!
[-] The following 1 user Likes agent_kith's post:
  • Snoopy8
Reply

#5
Further adventures with cpuset; information may be useful for the further extension of cpuset within SnakeOil.

Have a 4 CPU Intel NUC and used this script for my mpd setup
 
Code:
#
# set up cpusets
#
cset set -c 0 -s system
cset set -c 1 -s irq --cpu_exclusive
cset set -c 2 -s player --cpu_exclusive
cset set -c 3 -s server --cpu_exclusive
#
# move music server and player threads
#
cset proc --move --toset=player --pid $(pgrep mpd) --threads
cset proc --move --toset=server --pid $(pgrep camilladsp) --threads
#
# move all unbound kernel threads to system cpuset, and usb irq to irq cpuset
#
cset proc -k -f root -t system
cset proc --move --toset=irq --pid $(pgrep irq/123-xhci_hc) --kthread --force
Mpd and CamiilaDSP were moved to individual CPUs using a command similar to this:
"cset proc --move --toset=player --pid $(pgrep mpd) --threads".

Ideally, to reduce latency, IRQs should be on a different CPU to the music threads. Some IRQs were bound to CPUs, but many of the IRQs were moved using the following:
  1. Edited /etc/default/irqbalance and set IRQBALANCE_BANNED_CPUS=C  (hex mask value, run in CPUs 0,1 not in CPUs 2,3). 
  2. Alternately, disabled or uninstalled IRQBalance and more IRQs will run on the first CPU (CPU0)
  3. Used "cset proc -k -f root -t system" to move some more IRQs
  4. Used "cset proc --move --toset=irq --pid $(pgrep irq/123-xhci_hc) --kthread --force" to move USB IRQ to its own CPU.

Some music threads could not be moved eg. Minimserver is Java based. These music threads should be started within a cpuset using a command similar to this:
"cset proc -s server -e /bin/bash -- /var/www/players/minimserver/start.sh"

Some useful information on cpusets
https://documentation.suse.com/sle-rt/15...model.html
https://documentation.suse.com/sle-rt/15...puset.html
[-] The following 2 users Like Snoopy8's post:
  • hkphantomgtr, Valery
Reply

#6
Part III.   Had the following CPUsets (Intel NUC has 4 CPUs)
 
Code:
$cset set -l
cset:
         Name       CPUs-X    MEMs-X Tasks Subs Path
------------ ---------- - ------- - ----- ---- ----------
         root        0-3 y       0 y   207    4 /
       player          2 y       0 n     7    0 /player
       system          0 n       0 n    31    0 /system
       server          3 y       0 n     5    0 /server
          irq          1 y       0 n     1    0 /irq

The player CPUset ran the following mpd processes:
 
Code:
$ cset proc -l -s player
cset: "player" cpuset of CPUSPEC(2) with 7 tasks running
USER       PID  PPID SPPr TASK NAME
-------- ----- ----- ---- ---------
root       401     2 Sr55 [irq/127-eth0]
root      1082     1 Sr75 [mpd]
root      1083     1 Sr75 [io]
root      1084     1 Sr75 [rtio]
root      1090     1 Sr75 [player]
root      1091     1 Sr75 [decoder]
root      1379     1 Sr75 [output]

The server CPUset was running DSP:
 
Code:
$ cset proc -l -s server
cset: "server" cpuset of CPUSPEC(3) with 5 tasks running
USER       PID  PPID SPPr TASK NAME
-------- ----- ----- ---- ---------
root       589     1 Sr80 [camilladsp]
root       628     1 Sr80 [camilladsp]
root       631     1 Sr80 [camilladsp]
root       632     1 Sr80 [AlsaPlayback]
root       633     1 Sr80 [AlsaCapture]

The IRQ CPUset was running the USB output:
 
Code:
$ cset proc -l -s irq
cset: "irq" cpuset of CPUSPEC(1) with 1 task running
USER       PID  PPID SPPr TASK NAME
-------- ----- ----- ---- ---------
root        87     2 Sr85 [irq/124-xhci_hc]

Note that processes had increasing proprities down the music chain. IRQ for Ethernet: 55, feeding Mpd: 75, then processed by CamillaDSP:80 and output to IRQ for USB: 85. This ensured that downstream processes were always ready to accept inputs.

With IRQBalance removed, IRQs ran in CPU0 (System CPUset). The USB IRQ, which was the most CPU intensive task, ran on its own CPU. The Ethernet IRQ, being the input for mpd, ran on the same CPU. Further optimisation was done by moving some kernel threads to CPU0 using "cset proc -k -f root -t system".

In addition, had a 5.10 RT Kernel (built my own), did BIOS tweaks (removed hyperthreading, virtual machine settings, unnecessary ports & features) and ran DSP (convolution filters).

With mpd playing music, ran "cyclictest -t4 -p 90 -N -s -i 10000 -l 10000 -q" twenty times, with the following results.

   

Maximum latencies were typically 5 to 7 µs, which is excellent. CPU 0 had three 8 µs and one 10 µs max latency, which is not an issue because it ran everything not directly associated with music playback. However, CPU 1 (USB IRQ) and CPU 2 (mpd) had one 9 µs max latency, which suggests there could be room for improvement?  Will discuss things that require further investigation in a follow up post.

In summary, delighted with the tweaks that delivered SQ improvement for SnakeOil.
[-] The following 2 users Like Snoopy8's post:
  • hkphantomgtr, agent_kith
Reply

#7
A note for AK when CPUset capabilities are further enhanced.  Using Ubuntu 20.04, SnakeOil 1.2.9

With Upmpdcli, used the following to shift it
 
Code:
cset proc --move --toset=player --pid $(pgrep upmpdcli) --threads

The upmpdcli processes are successfully shifted to the player CPUset
 
Code:
$ cset proc -l -s player
cset: "player" cpuset of CPUSPEC(2) with 16 tasks running
USER       PID  PPID SPPr TASK NAME
-------- ----- ----- ---- ---------
upmpdcli  1139     1 Sr60 [upmpdcli]
upmpdcli  1141     1 Sr60 [upmpdcli]
upmpdcli  1145     1 Sr60 [upmpdcli]
upmpdcli  1149     1 Sr60 [upmpdcli]
upmpdcli  1150     1 Sr60 [MHD-listen]
upmpdcli  1151     1 Sr60 [MHD-listen]
upmpdcli  1153     1 Sr60 [upmpdcli]
upmpdcli  1597     1 Sr60 [upmpdcli]
upmpdcli  1599     1 Sr60 [upmpdcli]
upmpdcli  1689     1 Sr60 [upmpdcli]
upmpdcli  5595     1 Sr60 [upmpdcli]
upmpdcli  5638     1 Sr60 [upmpdcli]
upmpdcli  5797     1 Sr60 [upmpdcli]
upmpdcli  5798     1 Sr60 [upmpdcli]
upmpdcli  5840     1 Sr60 [upmpdcli]
upmpdcli  5843     1 Sr60 [MHD-connection]

Mpd have the following PIDs and should be in player CPUset (see above)
 
Code:
$ pgrep mpd
1139
1163

However, Top shows that Mpd can run in all the CPUs
 
Code:
PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND         P
     87 root     -86   0       0      0      0 S  10.0   0.0   2:18.91 irq/122-xhci_hc 1
   1163 root     -61   0  424452  23240  11960 S   6.0   0.1   1:29.56 mpd             3
   1139 upmpdcli -61   0 1749136  10220   7796 S   0.7   0.0   0:02.04 upmpdcli        2
Processor 2 is for Player CPUset and upmpdcli is correctly running in it.  But Mpd is appearing in processor 3 in this snapshot (and different processors in other snapshots).

Note that Mpd running on its own can be assigned to a CPUset.   It appears that the embedded Mpd process in upmpdcli cannot be locked into a CPUset. No idea why...
[-] The following 1 user Likes Snoopy8's post:
  • agent_kith
Reply



Bookmarks

Possibly Related Threads…
Thread Author Replies Views Last Post
  cpuset and TOP JD1 4 2,635 03-Sep-2018, 06:29 PM
Last Post: JD1



Users browsing this thread:
1 Guest(s)

[-]
Our Sponsors

[-]
Welcome
You have to register before you can post on our site.

Username/Email:


Password:





[-]
Latest Threads
ZimaBoard 2x NIC, 2x SATA, 2x USB, 1x PC...
Last Post: agent_kith
26-Mar-2024 12:04 PM
» Replies: 13
» Views: 612
LMS Upgrade ?
Last Post: uglymusic
15-Mar-2024 09:14 PM
» Replies: 7
» Views: 149
Squeezebox shutdown
Last Post: TripleX
13-Mar-2024 07:34 AM
» Replies: 4
» Views: 71
MP3 Not Playing
Last Post: Coacharnold
10-Mar-2024 10:25 AM
» Replies: 3
» Views: 52
Fail to install myMPD
Last Post: hkphantomgtr
03-Mar-2024 08:45 PM
» Replies: 9
» Views: 195
Intel HFI Driver Can "Save Tons Of CPU C...
Last Post: hkphantomgtr
28-Feb-2024 09:15 PM
» Replies: 0
» Views: 41
Announcing Snakeoil Measurements 1.3.0
Last Post: agent_kith
26-Feb-2024 10:02 PM
» Replies: 94
» Views: 4137
More Problems with Squeezelite in 1.3
Last Post: Coacharnold
17-Feb-2024 09:47 PM
» Replies: 2
» Views: 69
Snakeoil Music server & Snakeoil its own...
Last Post: Snoopy8
17-Feb-2024 11:20 AM
» Replies: 8
» Views: 365
RoonBridge (64 bit only)
Last Post: scrarfussi
15-Feb-2024 08:26 PM
» Replies: 3
» Views: 4125

[-]
SnakeoilOS Mission Statement

Our mission is to create a free to use computer OS that is easy to install, intuitive to operate and play music that will connect and engage with you emotionally.

SnakeoilOS gives you the freedom to spend more time on listening, enjoying and exploring music. Wasting time on computers is now a thing of the past! Everything is constantly evolving/improving. Please check back often for updates.

If you like this project, do show your support with a small token donation. All donations collected will be used to run this website, and for purchasing new equipment for the project.


Powered By MyBB, © 2002-2024. Theme © Melroy van den Berg.