Where does the option to rotate the trees come from? I've never seen this technique used, but it seems to make a huge difference. Is there an option for this in the XGBoost package or is this something you added?
That's my addition. As I explained in the demo, this is an alternative way of randomizing tree structure.
Such techniques are rarely used (Rotational Forest is the most known example), and in practice may work well only in the case when you have many measurements of the same nature (temperatures / pressures / exchange rates).
If you have nice domain-specific feature engineering, rotating trees only spoils the model.
