HISTORY ingestProducts

VERSION	DATE		DESCRIPTION
1.0	2005-03-21	created from scratch
1.1	2005-06-14	set cdbfile column in QC1 database
1.1.1	2005-09-14	bug in file selection fixed
1.1.2	2005-12-22	produce listings in $DFO_LST_DIR
1.1.3	2006-01-23	prepared to run off-line
1.1.4	2006-01-26	improved error handling, email sent
1.1.5	2006-05-02	check consistency of NGAS host definition; dfosLog disabled
1.1.6	2006-11-08	select PRO CATG list made unique
1.2	2007-11-08	mode=SCIENCE supported; PRO.CLASS, PRO.TECH supported
1.3	2007-12-13	check for COMMENT keys before replacekey, check for raw file/header existence; create list of not successfully ingested frames [BW]
1.4	2008-01-24	writes call of cleanupProducts into job file
1.4.1	2008-05-13	fix for extremely large file numbers [BWo]
1.4.2	2008-06-18	query to verify archive ingestion changed [BWo]
1.5	2008-08-18	ingest ancillary fits files
1.5.1	2008-10-13	ingest ancillary files even if product dirs do not exist [BWo]
1.5.2	2008-10-29	ingests ancillary files only if not yet ingested; QC1 update logs only saved in case of error [BWo]
1.5.3	2008-10-30	bug fixed in line 513 [BWo]
1.5.4	2009-12-17	warning message for already ingested files improved; bug fix in case product directories are empty [BWo]
1.6	2010-04-21	enabled for parallel processing; check for existence and consistency of ancillary log files [BWo]
1.6.1	2011-05-23	check for REF_NGAMS_HOST disabled
1.6.2	2011-10-10	ING_FILE_NAME hard-coded
2.0 	2013-07-30	enabled for PHOENIX/IDP ingestion [RHa] (2013-07-30)
2.0.1	2014-01-07	ingestion of SCIENCE fits files turned off in non-PHOENIX mode
2.1	2014-05-08	parallel execution terminated; handling of ancillary files terminated; handling of IDPs modified
2.2	2014-08-18	enabled in new standard way for DFOS and PHOENIX; ALL_INS_SUM entries added for IDPs
2.2.1	2014-09-04	using new dpIngest instead of cdbIngest; replace calib..data_products by qc_metadata..qc_products, cdbfile --> origfile in queries
2.3	2014-09-15	new option -f to call fitsverify, for testing and ticket creation 
3.0	2015-07-20	modified to support phoenix v2.0
3.0.1	2015-07-27	minor improvement for PHOENIX_DAILY table (PHOENIX only)
3.0.2	2015-09-24	JOBS_CLEANUP filled both for DFOS and for PHOENIX
3.0.3	2016-04-27	small bug fix for PHOENIX_DAILY table
3.0.4	2016-06-22	check INGESTED for uncommented ERRORs only (PHOENIX only); optional CLEANUP_PLUGIN
3.1	2017-04-04	enabled for PHOENIX DEEP_MODE
3.1.1	2017-08-10	bug fix for MJD (:537)
3.2	2018-01-08	check for ENABLE_UPDATES; repeat dpIngest if failing
3.2.1	2019-05-22	add pseudo_date in ingestion to daily_idpstat in MCAL case [BWo]