Wednesday, September 25, 2019

Items for Kripa

OK, first off: I previously thought that my relegator code was written such that it would only work with tf2.0.  This is apparently not the case, and Kripa is able to run it on her machine with tf1.13.

Over the next two weeks, I'm going to rework the code to run with tf2.0, primarily because I would like to use eager execution.  This will make it easier to build custom loss functions that can have batch-dependent features.

For the time being, Kripa will work with tf1.13 and the old code.  I'd like her to do the following:

  • run (lots of) fits with the regressor.  Fits should be run with 10k train events, noise=0.2, angle=0.9.  Signal fractions should take three values per decade (log spaced) from 0.001 up to 0.5.  (That last value need not respect log spacing.)  Run 25 fits for each value combination.
  • For each fit, output important result parameters to a master results file (see the 'write_results' loop in master_moons.py).  You can make the file a pickle if you want to.
  • Add to the output file arrays of the decision function values and the resulting significance values so that you can make the plots in the next bullet.
  • Make plots of significance vs decision value threshold for each signal fraction.  It would be nice to do this as a "band" that includes the results of all fits for a given signal fraction.
For the time being, let's use $s / \sqrt(s+b)$ as our significance estimator/figure of merit.

No comments:

Post a Comment

Relegator update

Kripa has produced some really nice plots of significance vs decision function threshold for the regressor.  NICE. We also have plots of a...