New VB - Happy, but will MP3 mirror ever finish?

[pastim]

I installed a new VortexBox Midi yesterday, copied around 1TB files (> 19,000 tracks, some 16/44.1, some 24/96) from my Windows Box to it, scanned with LMS and all is well. That took around 7 hours. As a British TV Ad says 'It does what it says on the tin' so I'm very happy.

However, I set it to mirror my flacs (at 190Kbs) yesterday early evening, and it has still barely made a start at 16:00 today. Is it really going to take weeks to do this? I did a search through the forum and people talk about hours, not days or weeks. I really want to sort out backups but I don't want to do this until the MP3s are sorted out (I have several backups from my Windows box, so it isn't too urgent).

Secondly, does the VortexBox 'Get Cover Art' do anything at all? It tried my flacs for about 50 minutes and then said it was done. A quick look at the mirror log doesn't seem to show anything at all, and I don't know where it would have put the art even if it did find any. I don't know if location makes a difference. My tracks for non-classical are (mostly) stored by Album, but for classical they are stored by Composer/Work/Artist. I'm aware there are other ways to get Album Art, and to be honest I'm not really fussed about it, but I am trying many of the features so that I know what they do.

The only extras I have installed thus far are the Chrome App (it took me a few minutes to find out how to run it - using the curious 'App' symbol at the bottom of the new tab page that I have never bothered with before), and Webmin. Webmin is useful, but it took me ages to discover the 'options' feature on the Chrome Extension which allows one to change the default 'http' to 'https'. I ended up finding and editing the constants.js file before finding this option. Couldn't it be preset in the App?

[Ron Olsen]

I installed a new VortexBox Midi yesterday, copied around 1TB files (> 19,000 tracks, some 16/44.1, some 24/96) from my Windows Box to it, scanned with LMS and all is well. That took around 7 hours. As a British TV Ad says 'It does what it says on the tin' so I'm very happy.

However, I set it to mirror my flacs (at 190Kbs) yesterday early evening, and it has still barely made a start at 16:00 today. Is it really going to take weeks to do this? I did a search through the forum and people talk about hours, not days or weeks. I really want to sort out backups but I don't want to do this until the MP3s are sorted out (I have several backups from my Windows box, so it isn't too urgent).

VortexBox uses LAME to encode FLAC to mp3. This is very processor intensive; the time required depends greatly on the speed of your CPU. You can get a measure of the CPU speed of your VB by the following:

1. Login to your VB; see How To Login to Your VortexBox.

2. Enter

grep bogomips /proc/cpuinfo

This will give you a measure of the processor speed. On my VortexBox Appliance, this gives a value around 3,300 for a dual-core AMD E-350 processor.

3. You can use htop to see if there is anything else running on your VB that may be slowing things down. Enter

yum install htop

to install it, and

htop

to run it. When mp3 mirroring is running, you should see lame taking almost all the CPU time.

Secondly, does the VortexBox 'Get Cover Art' do anything at all? It tried my flacs for about 50 minutes and then said it was done. A quick look at the mirror log doesn't seem to show anything at all, and I don't know where it would have put the art even if it did find any. I don't know if location makes a difference. My tracks for non-classical are (mostly) stored by Album, but for classical they are stored by Composer/Work/Artist. I'm aware there are other ways to get Album Art, and to be honest I'm not really fussed about it, but I am trying many of the features so that I know what they do.

The VortexBox Get Cover Art feature is hit-and-miss. It works well for popular albums, but not so well for more obscure albums. I recommend using bliss to find and upgrade cover art on your VB. Install it from the VB GUI Upgrade page http://vortexbox/upgrade.php. See http://www.blisshq.com for more info. The first 100 fixes are free, then you pay for more. Well worth the cost, IMO.

If you want to get cover art yourself, go to http://www.albumartexchange.com. Find a 600x600 image that you like, download it, and place it in cover.jpg in the album directory. Highly recommended as the best source for high-quality cover art.

The only extras I have installed thus far are the Chrome App (it took me a few minutes to find out how to run it - using the curious 'App' symbol at the bottom of the new tab page that I have never bothered with before), and Webmin. Webmin is useful, but it took me ages to discover the 'options' feature on the Chrome Extension which allows one to change the default 'http' to 'https'. I ended up finding and editing the constants.js file before finding this option. Couldn't it be preset in the App?

The Chrome App is not an official part of VortexBox; it was developed by user divreg. Contact the author to request changes. See Chrome App Wiki Page for more info.

[pastim]

VortexBox uses LAME to encode FLAC to mp3. This is very processor intensive; the time required depends greatly on the speed of your CPU. You can get a measure of the CPU speed of your VB by the following:

1. Login to your VB; see How To Login to Your VortexBox.

2. Enter

grep bogomips /proc/cpuinfo

This will give you a measure of the processor speed. On my VortexBox Appliance, this gives a value around 3,300 for a dual-core AMD E-350 processor.

3. You can use htop to see if there is anything else running on your VB that may be slowing things down. Enter

yum install htop

to install it, and

htop

to run it. When mp3 mirroring is running, you should see lame taking almost all the CPU time.



The VortexBox Get Cover Art feature is hit-and-miss. It works well for popular albums, but not so well for more obscure albums.

My system is a Midi appliance. The reading I get from bogomips is:

bogomips : 3600.24
bogomips : 3600.02
bogomips : 3600.05
bogomips : 3600.08

However, this is quite meaningless to me since I don't know what other systems rate as. I know what mips are, but bogo? I did look it up on wikipedia, but am frankly little the wiser.

I ran 'top' before but the results were puzzling, to say the least. Lame was constantly reading over 100%, with flac at 7% or so, but the system was reported as > 70% idle. However, htop makes a lot more sense - thanks for that. Lame is indeed hogging one cpu. I don't really understand why it's so slow. My Windows based PC converts much, much quicker, and the 2 CPUs on that are rated at 2Ghz. I understand the Atom CPUs are rated at 1.8Ghz, so something in the Linux implementation of Lame running on VortexBox is seriously slow. I guess I'll have to manually convert on my PC and copy over, which will take me hours of manual effort.

Could I suggest you warn people in advance about this issue when migrating to VortexBox before they set mirroring going? My rather carefully laid plans have been somewhat blown apart by this. I thought it might take a day or 2, but at this rate it will be weeks.

As to the cover art, could you please tell me what I should expect to see from the VB efforts, however poor they may be. What am I looking for, and where? I have a fair few very common CDs (of the likes of 'Bridge over troubled water' and so on), but I see no cover.jpg in that folder, and the tracks don't appear to have been updated. Does the scanner just give up, or what? I'm aware there are other methods that may be better, but I'd like to understand this one first.

And finally...

One thing about forums that's been puzzling me (i.e. driving me nuts) is how people create the 'code' boxes. How is that done? When I look at the page source I see loads of html lines round each one, but I can't believe people type that lot in every time. What is the trick? I've scoured the Internet but never found an explanation.

[Ron Olsen]

Looks as if your VB has reasonably powerful processors. I have no idea why the lame encoding is taking so long. I presume your /storage directory is on an internal hard drive, and not an external NAS? What does

rpm -q lame

show?

You can change the flac-to-mp3 mirror process to use two threads instead of one. This should speeds things up. See http://vortexbox.org/threads/2952-FL...ll=1#post21205 for details. I also created a wiki page for this: Multi-Thread FLAC Mirror.

The VortexBox Get Cover Art function places cover art in cover.jpg in each album directory, if it finds any. You can rerun it by clicking the "Get CD Cover Art" button on the VB GUI Auto Ripper page http://vortexbox/ripstatus.php. Using bliss to find/update cover art will be much more effective.

Code boxes are generated by placing [ code ] at the beginning and [\ code ] at the end (without any spaces).

If you click "Go Advanced" when creating a post, you may also see a "#" icon which will let you select text, and then place it inside a code block.

[pastim]

Looks as if your VB has reasonably powerful processors. I have no idea why the lame encoding is taking so long. I presume your /storage directory is on an internal hard drive, and not an external NAS? What does

rpm -q lame

show?

You can change the flac-to-mp3 mirror process to use two threads instead of one. This should speeds things up. See http://vortexbox.org/threads/2952-FL...ll=1#post21205 for details. I also created a wiki page for this: Multi-Thread FLAC Mirror.

I'll try the 2 thread process in a while. I worked out the current process would take 26 days. I'm currently using foobar on my PC, running over the lan, to convert, and it is doing 500 or more an hour, which is a lot quicker, but a nuisance, since I'll have to move a lot of files around later.
The rpm shows

lame-3.99.3-23.fc16.i686

The VortexBox Get Cover Art function places cover art in cover.jpg in each album directory, if it finds any. You can rerun it by clicking the "Get CD Cover Art" button on the VB GUI Auto Ripper page http://vortexbox/ripstatus.php. Using bliss to find/update cover art will be much more effective.

I did finally find about 30 cover.jpgs scattered randomly amongst my folders, so at least I know it tried to worrk, even if it didn't do much.

Code boxes are generated by placing [ code ] at the beginning and [\ code ] at the end (without any spaces).

If you click "Go Advanced" when creating a post, you may also see a "#" icon which will let you select text, and then place it inside a code block.

And I should have guessed about the *code*, but not knowing any html I had tried <code>. Thanks a lot.

[Ron Olsen]

I think your slow processing times with lame are a result of the Atom processors you have on your VB. Here is a test comparing the old Intel Pentium 4 with an Intel Atom D510. It states that lame runs much faster on the P4: http://www.tomshardware.com/reviews/...op,2649-3.html

So it appears that Atom processors are not great when it comes to using lame for mp3 transcoding. I suggest trying 2 or 4 threads in the flac mirror script and see if that helps.

[pastim]

I think your slow processing times with lame are a result of the Atom processors you have on your VB. Here is a test comparing the old Intel Pentium 4 with an Intel Atom D510. It states that lame runs much faster on the P4: http://www.tomshardware.com/reviews/...op,2649-3.html

So it appears that Atom processors are not great when it comes to using lame for mp3 transcoding. I suggest trying 2 or 4 threads in the flac mirror script and see if that helps.

Will do - thanks.

Assuming it's the Atoms that are slow I think VortexBox may need to reconsider the CPUs they supply, or at least warn people, many of whom might end up in the same situation.

By the way, you asked if I was using an internal disk. I am. The Vortexbox-supplied 3TB 7200rpm drive - should be fast enough.

[Ron Olsen]

Try this experiment to see how fast lame is on your VB:

1. Copy a FLAC file to /tmp.

2. Decode the FLAC file to WAV:

flac -d file.flac

3. Encode the WAV to mp3 and time the operation:

time lame -V2 file.wav file.mp3

On my old 2 TB VBA with a single-core 1.6 GHz Intel 230 processor, lame takes around 45 seconds on a 4-minute test track.

On my new 9 TB VBA with dual-core AMD E-350 processors, lame takes around 20 seconds on the same test track.

All experiments with lame-3.99.3-23. It would be interesting to see the performance numbers on your VB.

If you are not happy with the performance of your 3 TB VBA, you should contact Andrew and see if you can return it and get something that will work better for you.

[pastim]

If you are not happy with the performance of your 3 TB VBA, you should contact Andrew and see if you can return it and get something that will work better for you.

Assuming I'll never have to do a complete transcode again I'm not unhappy with the box - far from it. It's just that it would have been good to know beforehand about this issue, so that I could have found another way round it before spending quite a bit of time on it, rather than after (and using up your own time as well).

I am, however, assuming that I won't encounter any other really really slow processes. If there any others you know of I'd be grateful for fore-warning. The VB seems to handle LMS well, including Custom Scans which on my faster PC running Windows take twice as long at the VB does.

[Ron Olsen]

I'm still curious to see how long it takes lame to encode a test track on your VBA, to see how it compares to my old and new VBAs. See my previous post for an experiment you can run.