So we needed a new target for compatibility. We agreed that aiming for compatibility with 95% of devices in use would be a reasonable target.
I have been using caniuse.com to check individual features against their usage stats. It's a really useful site!
But our policy shouldn't just be "any feature supported by 95% of browsers can be used": if two features each have 95% support, the set of browsers supporting both features could be anything between 95% and 90%. With more features, the intersection of all those sets could end up being much lower than our target, and make Numbas inaccessible for a significant number of people.
I looked around for something that would let me specify a collection of features and see what percentage of browsers support them all. I'm sure it exists, but I couldn't find it, so I spent a happy day making it.
The tool is online at somethingorotherwhatever.com/can-i-also-use.
I gathered MDN's compatibility data and caniuse.com's usage data, and presented it as a massive searchable tree of features you can tick off. The other half of the screen shows you which browsers support all of the selected features, and their relative usage and release dates. For each browser, you're shown which features force that particular version, so you know what to drop in order to be supported by older versions.
I've realised that I have to be quite careful about which features I tick when using this tool, particularly with respect to CSS: browsers usually silently ignore CSS rules that they don't support, so I should only tick features that are absolutely required for the page to function.
There's a function to download the set of selected features so you can documente what your project requires. I haven't added this yet, but next time I need to do this job I'll add a feature to load in that file, so I don't have to start again from scratch!