AtOM: Anything to Ogg and Mp3
This program is for you if
- you want to have your music collection in the highest quality possible
- you have devices that don't have enough storage to hold it or
- you have devices that support a limited number of formats
To satisfy that need, we take a "source" directory, inventory and read tags for every file therein. From that, we can create and maintain copies of said directory in other formats or quality. There is also the possibility of ignoring or simply copying files with certain mime-types or extensions.
We try to do this in a smart way, by only treating files that are new or changed since the last run.
Apart from transcoding from one format to another, AtOM can also change sample-rate, bitrate, and the number of channels! Say, for example, that you want to stream your music through icecast. In addition to having all the files in the same format, it will want a constant sample-rate and bitrate. You can have AtOM do that!
Here's what I have for my tests:
| Directory | Format | Sample rate | Bitrate | Channels | FAT32 compat. | ASCII | Size |
|---|---|---|---|---|---|---|---|
| 0-Full | Mixed | Mixed | Mixed | Mixed | No | No | 508G |
| 1-High | Opus | Same | 128 | Same | Yes | No | 101G |
| 2-Medium | Opus | Same | 64 | Same | Yes | No | 59G |
| 3-Small | Opus | Same | 32 | Same | Yes | No | 30G |
| 4-MP3 | MP3 | 44100 | 128 | 2 | Yes | Yes | 114G |
- URL: https://framagit.org/atom/AtOM/
- Author: Vincent Riquer vincent+prog.atom@riquer.fr
- Copyright/left: 2012-2013,2015,2025 Vincent Riquer - GPLv3
Dependencies
Required:
Optional:
Some features will be disabled when not present.
- vorbis-tools
ogginfo(Ogg Vorbis metadata)oggenc(Ogg Vorbis encoding)
- opus-tools
opusinfo(Opus metadata)opusenc(Opus encoding)opusdec(Opus decoding)
- LAME MP3 Encoder
lame(MP3 encoding)
- FLAC
metaflac(FLAC metadata)
- Musepack
mpcdec(Musepack decoding)
- FFmpeg
ffprobe(ID3v2, Musepack, Windows Media and video metadata)ffmpeg(Windows Media and video decoding)
Using the software
Installation
- Clone the repository:
git clone https://framagit.org/atom/AtOM.git - run
./configure && make && sudo make install
./configure takes an optional prefix: ./configure --prefix=/usr. Defaults to /usr/local.
Configuration:
On first run, AtOM will ask a set of questions to help you create a
configuration file.
You can run atom -S at any time to re-run the setup. It will be prefilled with
your current configuration.
If, however, you still want to make changes manually, please read doc/config. There are a lot of comments in the generated config file too.
Preparing data:
Nothing specific needs to be done. You can edit your tags, rename files, move them around how you see fit. However, make sure you setup your tag editor to do update the files' timestamps: though it was initially plan to make this optional, using checksums or tags, it was abandoned due to the huge amount of IO required.
Running:
Make sure your configuration is correct by running
$ atom -C
This will produce a human-readable dump of your current configuration. If all settings are correct, simply run atom with no argument. Go get a beer. Meet some friends. Go to bed. Depending on the size of your collection, the first run can take hours, even days. After adding/tagging/renaming/deleting files, just re-run atom. It should be much faster this time, as only changed data will be treated.
If, for whatever reason, you need to force the regeneration of a destination, after changing the quality settings for example, run
$ atom -F <destination name>
Running as a cronjob:
If you want to run AtOM as a cronjob, atom -q will give you a cleaner output,
more suitable for mail or logfile output. You may also want to limit the size of
each batch with -B <batch size>. AtOM will not create or update more than
<batch size> destination files.
For example:
#m h dom mon dow command
0 5 * * * atom -B 1000 -q
Full options list
Please refer to atom -h, as this will always be the most up to date.
Technical details
I. Source scan
After reading its configuration file, AtOM uses find to get a list of all files in the source directory. Each file is checked against the database. If it's already there, and its last modification time is unchanged, the last_seen field is updated, and that's all. If its mtime has changed, mime-type scan is attempted. It is updated in the database, along with last_seen. If the file is new, its mime-type is scanned, and it is added to the database.
II. Obsolete files
Using the last_seen field, AtOM removes from destinations each files which are not present anymore in the source directory. AtOM never touches files not present in its database (unless there is a filename conflict, in which case your file WILL be overwritten). If you wish to clear unknown files from your destinations, have a look at toys/cleandestinations.
III. Reading metadata
AtOM then tries to read metadata from each new or changed file. It also re-reads
metadata from files scanned with an older version of AtOM, if the parser for
that format has changed. The actual data read depends on the format, but at the
very least, AtOM should identify the sampling rate, bitrate and number of
channels. Unknown file types are scanned with ffprobe, so you may still have
some luck, depending on your FFmpeg setup.
IV. Task creation
For every destination files having their last change field different from their corresponding source file entry, we create one or more tasks, to generate or update (overwrite) the destination file. AtOM tries to generate as few tasks as possible, by reusing intermediate files wherever feasible. E.g. if you define two destinations, only differing by the encoding format, we will create only one decoding task, and two encoding tasks. On average, the number of generated tasks shall always be less than 2n (where n is destinationsfile count), unless each of them uses different sampling rate/normalization parameters.
Files matching the format, sampling-rate, channel count and bitrate constraints are copied (symlinked where possible) during that stage ("immediate copies"). (also see higher-than setting).
The steps required for each file depend on the format and destination parameters (resampling, aso.). Basically, one destination file requires 1 (if reusing decoded/resampled file) to 3 (if format can't be decoded using sox and resampling/normalization is required) tasks.
V. Running tasks
V.1 Running tasks
Progress display:
L:<current load>/<max-load> W:<active workers>/<concurrency> T:<last task>/<task count> (F:<failed tasks>) <pct>% <remaining time> (A:<average task duration>s/task) ETA:<estimated time of arrival>
V.2 Renaming files
If rename pattern (or FAT32 compatibility) for one or more destinations has changed, files already transcoded will be renamed. Otherwise, this step is skipped.
VI. Copies
During that stage, files which mime-types matched copy_mime-type directives are copied (symlinked where possible) to the destination. When a rename pattern is defined and impacts path, files that are not in the same directories as files which have been successfully transcoded are ignored, as AtOM cannot guess what the destination path should be. Otherwise, files are copied with their name and path unchanged.
VII. Obsolete files 2
Whenever a file is transcoded, if it was already present in the database but its name changed, following a rename pattern change, the old file is removed during that stage.
Toys
AtOM requires a database to function. Now that we have a database containing
various information about our media files, why not use it?
AtOM comes with a small set of tools in the toys/ directory. These are
documented in toys/README.
Shameless Self Promotion
I am the author of free (Creative Commons CC-By-SA) music which you can stream for free, or buy to get high quality and bonuses from Bandcamp. If you like electronic music taking its inspiration from Trance, Drum & Bass, Ambient and (rarely) Free Jazz, please check it out! Downloads are available in FLAC, Ogg, MP3, and more, and includes the "source code" (sequencer files and the likes) for most tracks. I am receiving 80% of the money you'll spend, so you won't be feeding some greedy BigCorp producer or distributor. And if you don't like it, you can still spread the word to friends who may like. You can see this as a way to thank me for this piece of code.
Legal
Some of the format and/or tool names cited above are trademarks belonging to their rightful owners. AtOM and its authors are not linked in any way to those companies or individuals. Said companies do not endorse nor support AtOM in any way.