mc toolchain working on nodes, not cluster

I have the mc toolchain working on the farm nodes.  Processed a few batches of 100 sl_mu events with p4vert, p4only and p pi- reconstruction hypotheses.  The root trees output look good.

I also rebuilt my local halld_recon, which took several hours.

Goal for today is to understand why mc recently quit working on the cluster.  Recent output files show that hdgeant4 seems to be working OK (produces a reasonably sized output file).  Next step in chain is mcsmear, and then hd_root. 

UPDATE: I think that I fixed most of the problem. Added flags to mcsmear call in kL_mc_job_G4.py (and similar for ppim_fastpi script):

smear_cmd = 'mcsmear -PJANA_CALIB_CONTEXT="variation=mc" -PJANA:MAX_RELAUNCH_THREADS=10 '

Code now runs for sl_mu on the cluster.  Deleting all previous MC files and rerunning.