[ale] Lack of speed up

[Jim Kinney]

This sounds like a hypervisor putting all the load on as few cores as
possible. But you don't specify what is providing the data. That IO is
easily a bottleneck as it's shared across physical and VM. As the datasets
are not super huge, make sure you have plenty of RAM on the VM and on 8
separate terminals copy the dataset into a shell variable. Use that as the
input for an 8-instance run. The second through eighth instance should load
faster as the first takes the hit on libs and code load.

-- 
James P. Kinney III

Every time you stop a school, you will have to build a jail. What you gain
at one end you lose at the other. It's like feeding a dog on his own tail.
It won't fatten the dog.
- Speech 11/23/1900 Mark Twain


*http://heretothereideas.blogspot.com/
<http://heretothereideas.blogspot.com/>*

On Sun, Apr 27, 2025, 3:47 PM Leam Hall via Ale <ale at ale.org> wrote:

> Hey all, I'm mildly stumped by a lack of performance issue, and could use
> insight.
>
> A friend was running Hyper-V on a Windows laptop, and spun an Ubuntu image
> with 8 cores to run some Python based analysis on two datasets. The
> datasets were roughly 150 and 190 MB respectively, and running one command
> line with & took about three hours. All eight cores were roughly 45%
> utilized, though with Hyper-V I'm not sure if that's cores assigned to the
> Ubuntu instance, or on the host machine. There was no memory swapping, and
> top showed one Python process.
>
> He wanted things a bit faster, so I suggested opening a second terminal
> and running the command on each dataset separately, not in series. The load
> on all 8 cores went up about 15%, there were two Python processes, but the
> processes still took 3 hours.
>
> Is this a Hyper-V based issue, or something else?
>
> Thanks!
>
> Leam
>
> --
> Software Engineer         (reuel.net/career)
> Scribe: The Domici War    (domiciwar.net)
> Coding Ne'er-do-well      (github.com/LeamHall)
>
> Between "can" and "can't" is a gap of "I don't know", a place of
> discovery. For the passionate, much of "can't" falls into "yet". -- lh
>
> Practice allows options and foresight. -- lh
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> https://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.ale.org/pipermail/ale/attachments/20250427/61937890/attachment.htm>