outdisk/fits

[Klaus Banse <kbanse@eso.org>]

Dear Michel Laget,


     due to the selection of 400 000 rows in your table tmp0.tbl you have
created a descriptor SELIDX of 400 001 integer elements 
(total no. of selections + no.s of selected rows).
And COPY/TABLE dutifully copies also all descriptors of tmp0.tbl to
tmp1.tbl.

The long execution time of outdisk/fits then results from reading that
descriptor and writing it out to FITS, for SELIDX you create 400 001
lines in the FITS header...
And as you also pointed out, even READ/DESCR gives up on such a big 
descriptor. Also SHOW/DESCR is of no big help, because
Mid00 015: show/desc tmp1.tbl selidx
frame: tmp1.tbl  (data = R4)
name: SELIDX          (NoSel, selected rows ...)
type: integer         no. of elements:*****

Currently, only the listing of the internal descriptor directory of tmp1.tbl
displays the real size:
Mid00 016: read/desc tmp1.tbl descriptor.directory
frame: tmp1.tbl  (data = R4)
Descr_dir_extension 1:
-----------------------
Descr_dir_entry #0 uses 2 basic blocks
Name = DESCRIPTOR.DIRECTORY
   Type = C, No_elem = 7500, Bytes_per_elem = 1, unit = 0
   Start block, Index for data = 2, 1
Descr_dir_entry #1 uses 2 basic blocks
Name = TBLENGTH
   Type = I, No_elem = 9, Bytes_per_elem = 4, unit = 0
   Start block, Index for data = 14, 349
Descr_dir_entry #2 uses 2 basic blocks
....

Name = HISTORY
   Type = C, No_elem = 240, Bytes_per_elem = 1, unit = 0
   Start block, Index for data = 14, 447
Descr_dir_entry #9 uses 2 basic blocks
Name = SELIDX
   Type = I, No_elem = 400001, Bytes_per_elem = 4, unit = 0
   Start block, Index for data = 14, 470
   Help text = 24 chars, Start block, Index = 34408, 199
Descr_dir_extension 2:
-----------------------
Descr_dir_extension 3:
-----------------------
Descr_dir_extension 4:
-----------------------
Descr_dir_extension 5:
-----------------------
Descr. directory:
size = 7500, in_use = 600 (chars)
no_entries = 10, no_descr = 10 (incl. direc)


Now, get rid of the descriptor via

DELETE/DESCR tmp1.tbl selidx

and then the outdisk/fits command is as fast as before and you save yourself
a huge FITS header.


Cheers,
Klaus Banse


P.S. That leaves us with the question of what to do with such a huge SELIDX
descriptor resulting automatically from the SELECT/TABLE command.
Since the logical expression in that command can fill up all paramters
P2, ..., P8 there is no "space" for a YES/NO flag for the descr. creation.
Should we just put in a limit of, say, 100 000 elements or leave it to the
discretion of the user to later on delete that descr. if it's not needed?