Where does the option to rotate the trees come from? I've never seen this technique used, but it seems to make a huge difference. Is there an option for this in the XGBoost package or is this something you added?
That's my addition. As I explained in the demo, this is an alternative way of randomizing tree structure.
Such techniques are rarely used (Rotational Forest is the most known example), and in practice may work well only in the case when you have many measurements of the same nature (temperatures / pressures / exchange rates).
If you have nice domain-specific feature engineering, rotating trees only spoils the model.
Thanks axelr. Perhaps you could have a blog post in the future on how you made the GB playground using TF playground? It would be really helpful for people learning and teaching concepts on machine learning!
Hi,
I wasn't using tensorflow demo code - it is too demo-specific and written in typescript (which is nice, but ...).
As of programming, I coded gradient boosting in an hour or so - as I've done that many times before in python, this was an easy part. Later ~2 hours were spent on adding parameters / features to GB.
Most of the time was spent on proper visualization (different plotting libs are slow or too complicated, so I finished up with doing everything with canvas using some tricks), some code optimization and correct layout + support of mobile devices (so demo should be working on most mobile devices). And this took a week or even more.
To sum up, time is usually spent on something ridiculous (wrong event handling, wrong plotting, wrong layout), not on math or machine learning :)
