The EMMS filter system
- The EMMS filter system
The EMMS filter system
EMMS is the Emacs Multi Media System, and this article is about the new filtering and searching system that just became available with version 23 of EMMS.
I hope that this post will ease the introduction to such a big change in the EMMS browser.
As an introduction this post is already too long. However there is extensive documentation in the EMMS manual which goes into the details of every aspect of the filter system. It is, by far, the largest section in the EMMS manual. Bad or missing documentation strings are personal pet peeve, so I have tried to ensure that everything is well documented in every way. I’m going to forget stuff. I’ll need to look it up too.
Introduction
This is an introduction to the new EMMS filter system that has recently been merged into version 23 of EMMS. The new filter system completely changes how filters and searches are defined and used within EMMS. The system is backward compatible with the old EMMS browser code if you happen to have any filters or searches that you created through the browser’s previous system.
I came to EMMS with the jaundiced eye of a tango DJ and teacher. Tango music is particular and demanding. Historically metadata and recording quality has been terrible. All of the classic tango music we have today has been restored from old shellacs or vinyl. Tango music forces one to become a researcher, historian, restorer, and archivist. A good search and filter system is of utmost importance.
The Browsers Way.
There were two methods of searching in the EMMS browser. A search and a filter. They did not work together. Both operated solely on the main database file. A search would result in a temporary database cache with the records indicated by the search. A filter was simply a rendering of the main database where some records were filtered out. This was, from my point of view, unusable.
Backward Compatibility with the EMMS-Browser
As I started ‘fixing’ it a major concern was backwards compatibility with the browser. The way the browser defined filters and searches, as well as it’s behavior became fundamental driving forces in the new system as it emerged.
The old way of making filters, and giving them names connects directly into emms-filters with a thin function layer in the browser. The macro, emms-browser-make-filter is a wrapper around emms-filters-make-filter.
Searches are also direct connections into emms-filters.
Some things like the boiler plate code the browser needed to create a default filter are no longer needed. If you have this code, simply delete it. emms-browser-set-filter no longer exists.
(emms-browser-make-filter "all" 'ignore)
;; Set "all" as the default filter
(emms-browser-set-filter (assoc "all" emms-browser-filters))
If you wish to have a default filter which is always active you can do this. The hard-filter and pop are optional, they push a new database cache on the cache stack so that all future searches and filters will operate on this filtered cache rather than the full database cache.
;; Push a filter on the filter stack.
(emms-filters-push "myfilter")
;; Create a new cache from the filtered tracks and push it
;; push it on the cache stack
(emms-filters-hard-filter)
;; Pop the filter as it is no longer needed.
(emms-filters-pop)
I mostly succeeded at keeping the browser’s API and behavior, but my suggestion to you is to toss out any filter and search code you have for it and start over with this new system. Its so much easier and better.
The New Filter System
There are still searches and filters. They work together, they both use the same ‘filters’. Filtering is displaying a filtered cache. Searches use filters to create smaller caches which can then be filtered or searched again. Filters can be ‘hardened’ to create new caches as well.
Filters can be built interactively using the filter stack. They can also be defined as data in elisp code.
The old browser searches have been expanded a bit, there are a few more than before. They are handy but somewhat limited. The searches work as they always did, creating a new cache. They filter the cache, and push a new cache on the cache stack.
In the new system this amounts to these steps.
- Prompt for the filter
- Push the filter
- Hard-filter to push a new cache
- Pop the filter, leaving just the new cache as evidence.
The Filter Factories
The system uses filter factories to create anonymous lamba filter functions. Factories are registered in the system with a name, function, and a prompting and coercion definition for it’s expected values. Filter factories return a filter function with lexically bound values.
There are plenty of factories already defined. In reality most factories are derived from one of the more fundamental factories and are much easier to write.
For the most part there is no reason to make more factories. There may be something the system can’t do, but I haven’t thought of anything that I have envie for yet.
Here is the Genre factory definition and registration which uses a partial of the more complex field-compare factory.
(emms-filters-register-filter-factory
"Genre"
(apply-partially 'emms-filters-make-filter-field-compare
'string-equal-ignore-case 'info-genre)
'(("Genre: " (:string . nil))))
The first parameter is the filter name, the second is the factory function, and the third is the prompting definition so that it may be used interactively.
The Filters
All filters can be described as data but take life as functions. They are organized in a ring belonging to the filter factory that created them. Any filters created interactively will remain in their factory’s list until emacs is restarted. A number of filters are also provided. Defining more filters in your configuration simply adds to what is already available.
Here is the data definition for the Genre filter named Vals.
;; factory, name, value
("Genre" "Vals" "vals")
The Stacks
Managing and building the search caches and filters is done through two stacks. Each has the usual functionalities of a stack and possibly a few suprising ones, like clear, squash, smash, and swap-pop.
The Search Cache Stack
The search cache stack always has the main EMMS database at its base. Any filtering or searching is always applied to the top cache on the stack.
Each new search, or hardened filter, puts another cache on the stack.
Caches can be popped, stashed or stashed and popped, A stash can be pushed as many times as you wish. The stash is currently ephemeral for the session.
The Filter Stack
The filter stack is an interactive multi-filter builder. Filters can be combined interactively on the stack to create more complex ‘multi-filters’. The usual logic for queries can be created by choosing the correct flavor of push. The filter stack has a few push flavors, push, push-and, push-or, and push-not.
The current filter can be saved at anytime as data so that it may added to your collection of pre-built filters on startup.
A Picture is Worth…
Here is an image of a hydra that I like to use for the filters and caches. it includes a sort of dashboard that shows at a glance what is going on in the filter system. It shows the same info available through ‘emms-filters-status’. It also has all the commands you might ever need which are also directly in the browser’s mode keymap.
This first example has built a filter for an album-artist of Di Sarli, a year range of 1940 - 1946 and has a ring-filter setting of ‘Tango’.
The evolution of the filter being built can be seen on the filter stack. FIrst it was Di Sarli, then I added an ‘and’ of the year range. I had also added a genre filter for tango, but decided to show the ring filter in use instead, so I popped the ‘and Tango’ filter to have what is here.
The resulting list of tracks is Di Sarli, 1940-1946, Tango. Podesta sang a lot of tangos.
This example shows that I turned the previous filter into a cache with ‘hard-filter’ putting a new cache on the cache stack. I then cleared the filter stack as it was no longer needed. The cache holds only tracks of Di Sarli from 1940-1946. I set the ring filter to Vals this time.
The ring filter defaults to no filter* which is always a choice no matter what other filters are there. In my case I only have ‘tango’, ‘vals’ and ‘milonga+’.
Here we have a cache, an empty filter stack and a ring filter of Vals.
The resulting list of tracks Di Sarli, 1940-1946, Vals.
Getting Information
The filter sytem has quite a few functions that give information about itself. The primary function is ‘emms-filters-status-print’. Others are all prefixed with emms-filters-show- It is relatively easy to see what is inside the system.
Here are 2 filter status outputs. The first is the same as the last pair of images.
Ring: Vals
Meta Filter: nil
Filter stack:
Cache stack:
Di Sarli && 1940-1946
This one I built a filter for Album-artist, year-range, and genre.
Ring: no filter
Meta Filter: ("Di Sarli && 1940-1946 && Tango " ("Di Sarli") ("1940-1946") ("Tango"))
Filter stack:
Di Sarli && 1940-1946 && Tango
Di Sarli && 1940-1946
Di Sarli
Cache stack:
Filters as Data.
Creating new filters is very easy to do in code. Here are some examples which I hope are self explanatory.
Filters are always defined by their factory name, their name, followed by the parameters needed by the factory to create the filter.
Make Filter
In this case, the Genre factory, shown above, takes only a string value.
The Year range factory takes two 4 digit year values.
;; Factory Name Values
(emms-filters-make-filter "Genre" "Vals" "vals")
(emms-filters-make-filter "Year range" "1929-1937" 1929 1937)
Making Multiple Filters at Once.
It is much easier to build a group of filters together. This is how I do it in my configuration. It’s just a list of filter definitions.
Each filter is a list of factory name, filter name and factory parameters.
Multi-filters are a bit more complex, but remember that you can create them on the stack and ‘keep’ them if you are unsure how to write them at first.
Multi-filters are built from the names of other filters. A multi-filter is simply a list of lists. Each list is an AND, while the contents of each list are OR. A NOT is just a special list that starts with a keyword of :not.
(setq tango-filters
'(("Year range" "1900-1929" 1900 1929)
("Year range" "1929-1937" 1929 1937
("Year range" "1937-1942" 1937 1942)
("Year range" "1940-1946" 1940 1946)
("Year range" "1946-1958" 1946 1958)
("Year range" "1958-" 1958 3000)
("Directory" "tangotunes" "tangotunesflac")
("Genre" "Vals" "vals")
("Genre" "Tango" "tango")
("Genre" "Milonga" "milonga")
("Genre" "Condombe" "condombe")
("Genre" "Foxtrot" "foxtrot")
("Multi-filter"
"Milonga+"
(("Milonga" "Condombe")))
("Multi-filter"
"Milonga++"
(("Milonga+" "Foxtrot")))
("Album-artist" "Biagi" "Biagi")
("Album-artist" "Canaro" "Canaro")
("Album-artist" "Fresedo" "Fresedo")
("Album-artist" "Tanturi" "Tanturi")
("Album-artist" "Donato" "Donato")
("Album-artist" "Calo" "Calo")
("Album-artist" "Demarre" "Demarre")
("Album-artist" "Troilo" "Troilo")
("Album-artist" "Di Sarli" "Di Sarli")
("Album-artist" "Pugliese" "Pugliese")
("Album-artist" "OTV" "Orquesta Tipica Victor")
("Album-artist" "Carabelli" "Carabelli")
("Album-artist" "D'Arienzo" "D'Arienzo")
("Album-artist" "D'Agostino" "D'Agostino")
("Album-artist" "Lomuto" "Lomuto")
("Album-artist" "Laurenz" "Laurenz")
("Album-artist" "Rodriguez" "Rodriguez")
("Album-artist" "Malerba" "Malerba")
("Album-artist" "De Caro" "De Caro")
("Album-artist" "Firpo" "Firpo")
("Album-artist" "Salgan" "Salgan")
("Multi-filter"
"OTV+"
(("OTV" "Carabelli")))
("Multi-filter"
"1900-1937"
(("1900-1929" "1929-1937")))
("Multi-filter"
"1937-1946"
(("1937-1942" "1940-1946")))
("Multi-filter"
"Vals | milonga"
(("Vals" "Milonga")))
("Multi-filter"
"Vals 1900-1929"
(("Vals") ("1900-1929")))
("Multi-filter"
"Not Vals"
((:not "Vals")))
("Multi-filter"
"Vals or Milonga 1900-1937"
(("Vals" "Milonga+")
("1900-1929" "1929-1937")))))
(emms-filters-make-filters tango-filters)
Custom Filter Menus
Putting the filters in a variable allows using them to create a new category in the filter menus, in this case I’ll make a ‘Tango’ filter category. They will still show up under their respective factories menus as well. But this makes a central place for all of these filters so I can find them easily. There is another function that takes a list of filter names, if you wish to do it that way. This function simply retrieves the names from their definitions saving some effort.
(emms-filters-add-to-filter-menu-from-filter-list "Tango" tango-filters)
The Filter Ring
Once you have some filters, there is one additional place to use them. They can be placed in the filter ring. The ring is a short list of filters which can be quickly switched between with just the left and right arrows.
This was a feature of the Emms-browser and it is even easier now. The filter selected in the ring will be added to the filtering being done by the current filter on the stack.
The ring allows for quick switching between a few filters without interfering with the current filter.
For tango, this is quite convenient to switch between genre for a given search. When looking for music, I want to be able to switch quickly to a view that shows everything, just the valses, just the milongas or condombes, or Tango. Those filters are defined above.
They can be put in the filter ring like this.
(emms-filters-make-filter-ring '("Tango" "Vals" "Milonga+") )
Interactively Building Filters
The filter stack is, in essence, an interactive multi-filter builder. It creates the data definitions as you go, and each push results in a new filter function being assigned to be the current filter in use.
The filter stack is easy to use and you can explore all the factories and filters already available. If you build multifilters that you like you can ‘keep’ them. Keep saves the current filter to the file specified in the variable emms-filters-multi-filter-save-file.
(setq emms-filters-multi-filter-save-file "saved-emms-filters.el")
To start, simply push a new filter to the stack. You will be presented with a list of factories as well as new filter. Choosing new filter will give even more factory choices to choose from. Either way, choose a filter or fill in the prompts to make a new one.
From there, use push-and, push-or, push-not to build more complex filters. Doing another push will actually just create a second new filter on top. This filter will be active until you pop your way back down to the previous filter where you can continue where you left off.
Other stack functions are pop, swap, swap-pop, squash and smash. The current filter can be used to create a new cache on the cache stack with hard-filter. A filter can be saved as data to the save file at anytime with keep.
Interacting with the Cache Stack
The cache stack has the same basic commands to maniplate it as the filter. It doesn’t have keep but it does have stash, stash-pop and push. Stashes are only available during the session. Once stashed they remain in the stash and can be pushed back on to the cache stack as many times as you like. I think a folder of stashes would be nice to have, but one thing at a time.
Conclusion
Emms filters provides a super system for searching and filtering the media in the EMMS database. It’s built-in filter factories provide for just about everything you might ever want to search for.
The combination of it’s two stacks which grew from the old EMMS browser’s two separate ways of searching and filtering create a versatility that surprising and a real joy to use. The Emms filters system makes searching and exploring music fun.
