beets 1.5.0

This long overdue release of beets includes far too many exciting and useful features than could ever be satisfactorily enumerated. As a technical detail, it also introduces two new external libraries: MediaFile and Confuse used to be part of beets but are now reusable dependencies---packagers, please take note. Finally, this is the last version of beets where we intend to support Python 2.x and 3.5; future releases will soon require Python 3.6.

One non-technical change is that we moved our official #beets home on IRC from freenode to Libera.Chat.

Major new features:

• Fields in queries now fall back to an item's album and check its fields too. Notably, this allows querying items by an album's attribute: in other words, beet list foo:bar will not only find tracks with the foo attribute; it will also find tracks on albums that have the foo attribute. This may be particularly useful in the path-format-config`, which matches individual items to decide which path to use. Thanks to @FichteFoll. #2797 #2988
• A new reflink config option instructs the importer to create fast, copy-on-write file clones on filesystems that support them. Thanks to @rubdos.
• A new unimported lets you find untracked files in your library directory.
• The aura has arrived! Try out the future of remote music library access today.
• We now fetch information about works from MusicBrainz. MusicBrainz matches provide the fields work (the title), mb_workid (the MBID), and work_disambig (the disambiguation string). Thanks to @dosoe. #2580 #3272
• A new parentwork gets information about the original work, which is useful for classical music. Thanks to @dosoe. #2580 #3279
• bpd: BPD now supports most of the features of version 0.16 of the MPD protocol. This is enough to get it talking to more complicated clients like ncmpcpp, but there are still some incompatibilities, largely due to MPD commands we don't support yet. (Let us know if you find an MPD client that doesn't get along with BPD!) #3214 #800
• A new deezer can autotag tracks and albums using the Deezer database. Thanks to @rhlahuja. #3355
• A new bareasc provides a new query type: "bare ASCII" queries that ignore accented characters, treating them as though they were plain ASCII characters. Use the # prefix with list or other commands. #3882
• fetchart: The plugin can now get album art from last.fm. #3530
• web: The API now supports the HTTP DELETE and PATCH methods for modifying items. They are disabled by default; set readonly: no in your configuration file to enable modification via the API. #3870

Other new things:

• beet remove now also allows interactive selection of items from the query, similar to beet modify.
• Enable HTTPS for MusicBrainz by default and add configuration option https for custom servers. See musicbrainz-config for more details.
• mpdstats: Add a new strip_path option to help build the right local path from MPD information.
• convert: Conversion can now parallelize conversion jobs on Python 3.
• lastgenre: Add a new title_case config option to make title-case formatting optional.
• There's a new message when running beet config when there's no available configuration file. #3779
• When importing a duplicate album, the prompt now says "keep all" instead of "keep both" to reflect that there may be more than two albums involved. #3569
• chroma: The plugin now updates file metadata after generating fingerprints through the submit command.
• lastgenre: Added more heavy metal genres to the built-in genre filter lists.
• A new subsonicplaylist can import playlists from a Subsonic server.
• subsonicupdate: The plugin now automatically chooses between token- and password-based authentication based on the server version.
• A new extra_tags configuration option lets you use more metadata in MusicBrainz queries to further narrow the search.
• A new fish adds Fish shell tab autocompletion to beets.
• plugins/fetchart and plugins/embedart: Added a new quality option that controls the quality of the image output when the image is resized.
• plugins/keyfinder: Added support for keyfinder-cli. Thanks to @BrainDamage.
• plugins/fetchart: Added a new high_resolution config option to allow downloading of higher resolution iTunes artwork (at the expense of file size). #3391
• plugins/discogs: The plugin applies two new fields: discogs_labelid and discogs_artistid. #3413
• export: Added a new -f (--format) flag, which can export your data as JSON, JSON lines, CSV, or XML. Thanks to @austinmm. #3402
• convert: Added a new -l (--link) flag and link option as well as the -H (--hardlink) flag and hardlink option, which symlink or hardlink files that do not need to be converted (instead of copying them). #2324
• replaygain: The plugin now supports a per_disc option that enables calculation of album ReplayGain on disc level instead of album level. Thanks to @samuelnilsson. #293
• replaygain: The new ffmpeg ReplayGain backend supports R128_ tags. #3056
• plugins/replaygain: A new r128_targetlevel configuration option defines the reference volume for files using R128_ tags. targetlevel only configures the reference volume for REPLAYGAIN_ files. #3065
• discogs: The plugin now collects the "style" field. Thanks to @thedevilisinthedetails. #2579 #3251
• absubmit: By default, the plugin now avoids re-analyzing files that already have AcousticBrainz data. There are new force and pretend options to help control this new behavior. Thanks to @SusannaMaria. #3318
• discogs: The plugin now also gets genre information and a new discogs_albumid field from the Discogs API. Thanks to @thedevilisinthedetails. #465 #3322
• acousticbrainz: The plugin now fetches two more additional fields: moods_mirex and timbre. Thanks to @malcops. #2860
• playlist and smartplaylist: A new forward_slash config option facilitates compatibility with MPD on Windows. Thanks to @MartyLake. #3331 #3334
• The data_source field, which indicates which metadata source was used during an autotagging import, is now also applied as an album-level flexible attribute. #3350 #1693
• beatport: The plugin now gets the musical key, BPM, and genre for each track. #2080
• A new bpsync can synchronize metadata changes from the Beatport database (like the existing mbsync for MusicBrainz).
• hook: The plugin now treats non-zero exit codes as errors. #3409
• subsonicupdate: A new url configuration replaces the older (and now deprecated) separate host, port, and contextpath config options. As a consequence, the plugin can now talk to Subsonic over HTTPS. Thanks to @jef. #3449
• discogs: The new index_tracks option enables incorporation of work names and intra-work divisions into imported track titles. Thanks to cole-miller. #3459
• web: The query API now interprets backslashes as path separators to support path queries. Thanks to @nmeum. #3567
• beet import now handles tar archives with bzip2 or gzip compression. #3606
• beet import also now handles 7z archives, via the py7zr library. Thanks to @arogl. #3906
• plexupdate: Added an option to use a secure connection to Plex server, and to ignore certificate validation errors if necessary. #2871
• convert: A new delete_originals configuration option can delete the source files after conversion during import. Thanks to logan-arens. #2947
• There is a new --plugins (or -p) CLI flag to specify a list of plugins to load.
• A new genres option fetches genre information from MusicBrainz. This functionality depends on functionality that is currently unreleased in the python-musicbrainzngs library: see PR #266 _. Thanks to :user:aereaux`.
• replaygain: Analysis now happens in parallel using the command and ffmpeg backends. #3478
• plugins/replaygain: The bs1770gain backend is removed. Thanks to @SamuelCook.
• Added trackdisambig which stores the recording disambiguation from MusicBrainz for each track. #1904
• plugins/fetchart: The new max_filesize configuration sets a maximum target image file size.
• badfiles: Checkers can now run during import with the check_on_import config option.
• export: The plugin is now much faster when using the --include-keys option is used. Thanks to @ssssam.
• The importer's set_fields` option now saves all updated fields to on-disk metadata. #3925 #3927
• We now fetch ISRC identifiers from MusicBrainz. Thanks to @aereaux.
• metasync: The plugin now also fetches the "Date Added" field from iTunes databases and stores it in the itunes_dateadded field. Thanks to @sandersantema.
• lyrics: Added a new Tekstowo.pl lyrics provider. Thanks to various people for the implementation and for reporting issues with the initial version. #3344 #3904 #3905 #3994
• beet update will now confirm that the user still wants to update if their library folder cannot be found, preventing the user from accidentally wiping out their beets database. Thanks to user: logan-arens. #1934

Fixes:

• Adapt to breaking changes in Python's ast module in Python 3.8.
• beatport: Fix the assignment of the genre field, and rename musical_key to initial_key. #3387
• lyrics: Fixed the Musixmatch backend for lyrics pages when lyrics are divided into multiple elements on the webpage, and when the lyrics are missing.
• web: Allow use of the backslash character in regex queries. #3867
• web: Fixed a small bug that caused the album art path to be redacted even when include_paths option is set. #3866
• discogs: Fixed a bug with the index_tracks option that sometimes caused the index to be discarded. Also, remove the extra semicolon that was added when there is no index track.
• subsonicupdate: The API client was using the POST method rather the GET method. Also includes better exception handling, response parsing, and tests.
• the: Fixed incorrect regex for "the" that matched any 3-letter combination of the letters t, h, e. #3701
• fetchart: Fixed a bug that caused the plugin to not take environment variables, such as proxy servers, into account when making requests. #3450
• fetchart: Temporary files for fetched album art that fail validation are now removed.
• inline: In function-style field definitions that refer to flexible attributes, values could stick around from one function invocation to the next. This meant that, when displaying a list of objects, later objects could seem to reuse values from earlier objects when they were missing a value for a given field. These values are now properly undefined. #2406
• bpd: Seeking by fractions of a second now works as intended, fixing crashes in MPD clients like mpDris2 on seek. The playlistid command now works properly in its zero-argument form. #3214
• replaygain: Fix a Python 3 incompatibility in the Python Audio Tools backend. #3305
• importadded: Fixed a crash that occurred when the after_write signal was emitted. #3301
• plugins/replaygain: Fix the storage format for R128 gain tags. #3311 #3314
• discogs: Fixed a crash that occurred when the master URI isn't set in the API response. #2965 #3239
• spotify: Fix handling of year-only release dates returned by the Spotify albums API. Thanks to @rhlahuja. #3343
• Fixed a bug that caused the UI to display incorrect track numbers for tracks with index 0 when the per_disc_numbering option was set. #3346
• none_rec_action does not import automatically when timid is enabled. Thanks to @RollingStar. #3242
• Fix a bug that caused a crash when tagging items with the beatport plugin. #3374
• beet import now logs which files are ignored when in debug mode. #3764
• bpd: Fix the transition to next track when in consume mode. Thanks to @aereaux. #3437
• lyrics: Fix a corner-case with Genius lowercase artist names #3446
• parentwork: Don't save tracks when nothing has changed. #3492
• Added a warning when configuration files defined in the include directive of the configuration file fail to be imported. #3498
• Added normalization to integer values in the database, which should avoid problems where fields like bpm would sometimes store non-integer values. #762 #3507 #3508
• Fix a crash when querying for null values. #3516 #3517
• lyrics: Tolerate a missing lyrics div in the Genius scraper. Thanks to @thejli21. #3535 #3554
• lyrics: Use the artist sort name to search for lyrics, which can help find matches when the artist name has special characters. Thanks to @hashhar. #3340 #3558
• replaygain: Trying to calculate volume gain for an album consisting of some formats using ReplayGain and some using R128 will no longer crash; instead it is skipped and and a message is logged. The log message has also been rewritten for to improve clarity. Thanks to @autrimpo. #3533
• lyrics: Adapt the Genius backend to changes in markup to reduce the scraping failure rate. #3535 #3594
• lyrics: Fix a crash when writing ReST files for a query without results or fetched lyrics. #2805
• fetchart: Attempt to fetch pre-resized thumbnails from Cover Art Archive if the maxwidth option matches one of the sizes supported by the Cover Art Archive API. Thanks to @trolley. #3637
• ipfs: Fix Python 3 compatibility. Thanks to @musoke. #2554
• Fix a bug that caused metadata starting with something resembling a drive letter to be incorrectly split into an extra directory after the colon. #3685
• mpdstats: Don't record a skip when stopping MPD, as MPD keeps the current track in the queue. Thanks to @aereaux. #3722
• String-typed fields are now normalized to string values, avoiding an occasional crash when using both the fetchart and the discogs together. #3773 #3774
• Fix a bug causing PIL to generate poor quality JPEGs when resizing artwork. #3743
• plugins/keyfinder: Catch output from keyfinder-cli that is missing key. #2242
• plugins/replaygain: Disable parallel analysis on import by default. #3819
• mpdstats: Fix Python 2/3 compatibility #3798
• discogs: Replace the deprecated official discogs-client library with the community supported python3-discogs-client library. #3608
• chroma: Fixed submitting AcoustID information for tracks that already have a fingerprint. #3834
• Allow equals within the value part of the --set option to the beet import command. #2984
• Duplicates can now generate checksums. Thanks @wisp3rwind for the pointer to how to solve. Thanks to @arogl. #2873
• Templates that use %ifdef now produce the expected behavior when used in conjunction with non-string fields from the types. #3852
• lyrics: Fix crashes when a website could not be retrieved, affecting at least the Genius source. #3970
• duplicates: Fix a crash when running the dup command with a query that returns no results. #3943
• beatport: Fix the default assignment of the musical key. #3377
• lyrics: Improved searching on the Genius backend when the artist contains special characters. #3634
• parentwork: Also get the composition date of the parent work, instead of just the child work. Thanks to @aereaux. #3650
• lyrics: Fix a bug in the heuristic for detecting valid lyrics in the Google source. #2969
• thumbnails: Fix a crash due to an incorrect string type on Python 3. #3360
• fetchart: The Cover Art Archive source now iterates over all front images instead of blindly selecting the first one.
• lyrics: Removed the LyricWiki source (the site shut down on 21/09/2020).
• subsonicupdate: The plugin is now functional again. A new auth configuration option is required in the configuration to specify the flavor of authentication to use. #4002

For plugin developers:

• MediaFile has been split into a standalone project. Where you used to do from beets import mediafile, now just do import mediafile. Beets re-exports MediaFile at the old location for backwards-compatibility, but a deprecation warning is raised if you do this since we might drop this wrapper in a future release.
• Similarly, we've replaced beets' configuration library (previously called Confit) with a standalone version called Confuse. Where you used to do from beets.util import confit, now just do import confuse. The code is almost identical apart from the name change. Again, we'll re-export at the old location (with a deprecation warning) for backwards compatibility, but we might stop doing this in a future release.
• beets.util.command_output now returns a named tuple containing both the standard output and the standard error data instead of just stdout alone. Client code will need to access the stdout attribute on the return value. Thanks to @zsinskri. #3329
• There were sporadic failures in test.test_player. Hopefully these are fixed. If they resurface, please reopen the relevant issue. #3309 #3330
• The beets.plugins.MetadataSourcePlugin base class has been added to simplify development of plugins which query album, track, and search APIs to provide metadata matches for the importer. Refer to the spotify and the deezer for examples of using this template class. #3355
• Accessing fields on an Item now falls back to the album's attributes. So, for example, item.foo will first look for a field foo on item and, if it doesn't exist, next tries looking for a field named foo on the album that contains item. If you specifically want to access an item's attributes, use Item.get(key, with_album=False). #2988
• Item.keys also has a with_album argument now, defaulting to True.
• A revision attribute has been added to Database. It is increased on every transaction that mutates it. #2988
• The classes AlbumInfo and TrackInfo now convey arbitrary attributes instead of a fixed, built-in set of field names (which was important to address #1547). Thanks to @dosoe.
• Two new events, mb_album_extract and mb_track_extract, let plugins add new fields based on MusicBrainz data. Thanks to @dosoe.

For packagers:

• Beets' library for manipulating media file metadata has now been split to a standalone project called MediaFile, released as mediafile. Beets now depends on this new package. Beets now depends on Mutagen transitively through MediaFile rather than directly, except in the case of one of beets' plugins (in particular, the scrub).
• Beets' library for configuration has been split into a standalone project called Confuse, released as confuse. Beets now depends on this package. Confuse has existed separately for some time and is used by unrelated projects, but until now we've been bundling a copy within beets.
• We attempted to fix an unreliable test, so a patch to skip or repair the test may no longer be necessary.
• This version drops support for Python 3.4.
• We have removed an optional dependency on bs1770gain.