Monday, September 23, 2019

RELEGATOR update

Most of the code for investigating the relegator nn classifier on the moons+mass dataset is up and running.  The plan is to check increase in analysis significance, s/sqrt(s+b), for various architecture/loss function combos and various signal fractions.

Arch/loss combos:

  • regressor with binary CE loss
  • regressor with binary CE + significance loss
  • binary nn with categorical CE loss
  • binary nn with categorical CE + significance loss
  • relegator with modified categorical CE (no penalty for relegation class)
  • relegator mod CCE + signif loss
It would be ideal to test each on the same generated dataset, but this might take a lot more coding.  Perhaps some way to gen the dataset and THEN pass it into master_moons.py???

For the time being, we'll take the statistical approach --> run each MANY times, histogram the increases in signif.

So, next step is to add CCE+signif loss function capabilities to regressor and nn_binary, and to make the 5th model in the list above.  This means that we'll have a total of SIX models to test.

No comments:

Post a Comment

Relegator update

Kripa has produced some really nice plots of significance vs decision function threshold for the regressor.  NICE. We also have plots of a...