Should predict be robust to changes in column order?

[elipivo]

A quick test seemed to show that the predict method is not robust to reordering the column order, which was a bit surprising.

using DataFrames
using MLJ
import MLJDecisionTreeInterface.DecisionTreeClassifier as Tree

iris = DataFrame(load_iris());
y, X = unpack(iris, ==(:target); rng=123)

tree = Tree()
mach = machine(tree, X, y)
train, test = partition(eachindex(y), 0.7)
fit!(mach, rows=train)

# Returns false
predict(mach, X[test,[3,2,1,4]]) == predict(mach, X[test,[1,2,3,4]])

[ablaom]

You are right - you cannot reorder the columns. Actually, this kind of robustness has never been a hard requirement of the MLJ interface, which is unfortunate. Generally algorithms just coerce to a matrix and dump features names.

To fix this, MLJModelInterface.fit needs to save the names to cache part of output, to make them available to MLJModelInterface.predict for order-checking. The logic can get a little complicated if matrices instead of tables are also supported (in which case you can only check the number of columns is the same). This is not the case here, but it might be in the future.

PR welcome if you have the expertise and interest.