# Help message for `lbtree.cgi`

This CGI provides access to the 𝛼𝛿-tree indexes of all VizieR tables containing positional data.
For each table, the index is built from its main coordinates.
When necessary, these coordinates are converted to the equatorial FK5 system at Equinox J2000,
which is very close to ICRS. Proper motions are not taken into account.

The CGI supports three types of queries:

* `qregion` (GET): count the number of entries within a sky region (cone, zone, or HEALPix cell);
* `xmatch` (GET): perform a cross-match between the indexes of two tables; self-matching is also supported;
* `qlist` (POST): perform a cross-match between a user-uploaded table and a VizieR table index.


## Mode `qregion`

Count the number of entries within a sky region.

### GET parameters:

* `qtype=qregion`: to select the `qregion` mode
* `table=TABLE`: in which TABLE is a VizieR table identifier (e.g. `I/355/gaiadr3`)
* `region=cone|zone|hpx`: to select the skyregion. Here the parameters depending on the selected region:
    + `region=cone`: cone search selection
        - `ra=RA_DEG`: Right Ascension of the center of the cone, in degrees
        - `dec=DEC_DEG`: Decliantion of the center of the cone, in degrees
        - `sr=R_DEG`: search radius, in degrees
    + `region=zone`: zone selection
        - `ra.min=RA_DEG`: Right Ascension of the south-ovest corner of the zone
        - `dec.min=DEC_DEG`:  Decliantion of the south-ovest corner of the zone
        - `ra.max=RA_DEG`:  Right Ascension of the north-east corner of the zone
        - `dec.max=DEC_DEG`:   Decliantion of the north-east corner of the zone
    + `region=hpx`: HEALPix cell selection
        - `order=ORDER`: the HEALPix order of the cell
        - `ipix=IPIX`: the HEALPix index of the cell

### Limitations

An `overflow` occurs if the region contains 10_000_000 entries or more.
The query runs using a single thread.

### Examples

* [cone example](https://vizcat.cds.unistra.fr/cgi-bin/lbtree.cgi?qtype=qregion&table=I/355/gaiadr3&region=cone&ra=10.0&dec=-2.0&sr=30.0)
* [zone example](https://vizcat.cds.unistra.fr/cgi-bin/lbtree.cgi?qtype=qregion&table=I/355/gaiadr3&region=zone&ra.min=5.0&dec.min=-5.0&ra.max=15.0&dec.max=5.0)
* [HEALPix cell example]https://vizcat.cds.unistra.fr/cgi-bin/lbtree.cgi?qtype=qregion&table=I/355/gaiadr3&region=hpx&order=2&ipix=12)


## Mode `xmatch`

Perform a cross-match between the indexes of two tables; self-matching is supported, using the
same table name both for the left and right tables.

### GET parameter:

* `qtype=xmatch`: to select the `xmatch` mode
* `left=TABLE`: to select the left table(`TABLE` is a VizieR table identifier such as `I/355/gaiadr3`)
* `right=TABLE`: to select the right table(`TABLE` is a VizieR table identifier such as `I/355/gaiadr3`)
* `dmin=D_ARCSEC` (optional, default=0.0): minimal cross-match distance (in arcsec), useful for self cross-matches
* `dmax=D_ARCSEC` (optional, default=10.0): cross-match maximal distance, in arcsec
* `mode=best|all` (optional, default=all): select the nearest source or all sources
* `lmode=1|2` (optional, default=1): see "Limitations"
* `region=allsky|cone|zone|hpx` (optional, default=allsky): select sources of the left table in the given skyregion
    + `region=allsky`: select all left table sources
    + `region=cone`: select left table sources in a cone
        - `ra=RA_DEG`: Right Ascension of the center of the cone, in degrees
        - `dec=DEC_DEG`: Decliantion of the center of the cone, in degrees
        - `sr=R_DEG`: search radius, in degrees
    + `region=zone`: select left table sources in a zone
        - `ra.min=RA_DEG`: Right Ascension of the south-ovest corner of the zone
        - `dec.min=DEC_DEG`:  Decliantion of the south-ovest corner of the zone
        - `ra.max=RA_DEG`:  Right Ascension of the north-east corner of the zone
        - `dec.max=DEC_DEG`:   Decliantion of the north-east corner of the zone
    + `region=hpx`: select left table sources in a cone in a HEALPix cell
        - `order=ORDER`: the HEALPix order of the cell
        - `ipix=IPIX`: the HEALPix index of the cell
* `out=histo.txt|histo.svg|join.inner|join.left`: selection of the result
    + `out=histo.txt`: the result is a distance histogram, in txt format
        - `nbins=N (optional, default=20)`: number of bin in the histogram
    + `out=histo.svg`: the result is a distance histogram, in SVG format
        - `nbins=N (optional, default=20)`: number of bin in the histogram
    + `out=dxdy.txt`: the result is a (Delta x, Delta y) plot, in txt format
        - `width=N`:size of the image, in pixels
    + `out=dxdy.svg`: the result is a (Delta x, Delta y) plot, in SVG format
        - `width=N`:size of the image, in pixels
    + `out=join.inner`: the result is the inner join table, in CSV format
        - ` print.coo=true|false` (optional, default=false): print coordinates in addition to idnetifiers and distances
    + `out=join.left`: the result is the left join table, in CSV format
        - ` print.coo=true|false` (optional, default=false): print coordinates in addition to idnetifiers and distances

### Limitations

The CGI runs on a machine hosting several other CGIs, all of which are synchronous and accessible
to external users. To ensure that resources are shared fairly and that individual queries do not run
for too long, we impose some limits on query parameters. Those limits do not exists on the CLI version
of the tool.

NOTE: Please, **do not submit multiple jobs in parallel**. You are likely not the only user running
queries at a given time, and parallel submissions can unnecessarily consume shared resources.

* A single xmatch query uses 8 threads
    + note that at first execution, the limiting factor is often the disk access time
* `lmode=1`
    + `dmax` must be lower (or equal to) 60 arcsec (i.e. 1 arcmin)
    + the number of left entries cross-matches is limited to 5 million
    + the number of right entries returned by a left entries (`mode=all`) is limited to 10
* `lmode=2`
    + `dmax` must be lower (or equal to) 3600 arcsec (i.e. 1 deg)
    + the number of left entries cross-matches is limited to 50_000
    + the number of right entries returned by a left entries (`mode=all`) is limited to 1_000

### Examples

* Allsky xmatch of Tycho 2 (2.5 M rows) vs VSX (10 M rows), with TXT histogram output,
  see [here](https://vizcat.cds.unistra.fr/cgi-bin/lbtree.cgi?qtype=xmatch&left=I/259/tyc2&right=B/vsx/vsx&dmax=1.5&mode=all&region=allsky&out=histo.txt&nbins=30)
* Xmatch of 2MASS vs Gaia DR3 in a cone of 30.25 deg, with SVG histogram output,
  see[here](https://vizcat.cds.unistra.fr/cgi-bin/lbtree.cgi?qtype=xmatch&left=II/246/out&right=I/355/gaiadr3&dmax=1.5&mode=all&region=cone&ra=10.0&dec=-2.0&sr=30.25&out=histo.svg&nbins=100)
* Xmatch of 2MASS vs Gaia DR3 in a HEALPix cell, with inner join output,
  see [here](https://vizcat.cds.unistra.fr/cgi-bin/lbtree.cgi?qtype=xmatch&left=II/246/out&right=I/355/gaiadr3&dmax=3.0&mode=best&region=hpx&order=3&ipix=36&out=join.inner&print.coo=false)
* 2MASS self match in a zone of 20deg x 15deg
  see [here](https://vizcat.cds.unistra.fr/cgi-bin/lbtree.cgi?qtype=xmatch&left=II/246/out&right=II/246/out&dmin=0.0001&dmax=10.0&mode=all&region=zone&ra.min=5.0&dec.min=-5.0&ra.max=25.0&dec.max=10.0&out=histo.svg&nbins=100)
    + the zone contains 468_031 entries,
      see [here](https://vizcat.cds.unistra.fr/cgi-bin/lbtree.cgi?qtype=qregion&table=II/246/out&region=zone&ra.min=5.0&dec.min=-5.0&ra.max=25.0&dec.max=10.0)
* Xmatch XMM with SDSS DR16 at 5 arcsec, print the histo in SVG
  see [here](https://vizcat.cds.unistra.fr/cgi-bin/lbtree.cgi?qtype=xmatch&left=IX/59/xmm4dr9s&right=V/154/sdss16&dmax=5.0&mode=all&region=allsky&out=histo.svg&nbins=100)
* Xmatch XMM with SDSS DR16 at 3 arcsec, print the histo in SVG
  see [here](https://vizcat.cds.unistra.fr/cgi-bin/lbtree.cgi?qtype=xmatch&left=IX/59/xmm4dr9s&right=V/154/sdss16&dmax=3.0&mode=all&region=allsky&out=dxdy.svg&width=600)

## Mode `qlist`

Perform a cross-match between a user-uploaded table and a VizieR table index.

### POST parameters

The POST content must consist in a multi-part upload containig two distinct files.
* first, a JSON file containing the query parameters
    + its parameters name must be `args`
    + its Mime type must be `application/json`
* second, a CSV table made of 3 columns (no header): `id(u64),ra_deg(f64),dec_deg(f64)`
    + its parameters name must be `data`
    + its Mime type must be `application/octet-stream`

### Structure of the JSON file and examples

 * you can count the number of matches, either the grand total or for each entry:

```bash
{
  "qlist": {
    "right": "I/355/gaiadr3",
    "d_max": 3.0,
    "matches": {
      "count": {
        "limit": 30,
        "print": "sum"
      }
    }
  }
}
```

```bash
{
  "qlist": {
    "right": "I/355/gaiadr3",
    "d_max": 3.0,
    "matches": {
      "count": {
        "limit": 30,
        "print": {
          "list": {
            "print_coo": false
          }
        }
      }
    }
  }
}
```

* you can count the number of enrties having at list a match

```bash
{
  "qlist": {
    "right": "I/355/gaiadr3",
    "d_max": 3.0,
    "matches": {
      "nn": {
        "print": "count"
      }
    }
  }
}
```

* you can get an histogram of the match distancs for all matches or the nearest neighbour

```bash
{
  "qlist": {
    "right": "I/355/gaiadr3",
    "d_min": 0.0001,
    "d_max": 1.0,
    "matches": {
      "all": {
        "limit": 100,
        "print": {
          "histo": {
            "n_bins": 60,
            "fmt": {
              "svg": true
            }
          }
        }
      }
    }

```

```bash
{
  "qlist": {
    "right": "I/355/gaiadr3",
    "d_max": 3.0,
    "matches": {
      "nn": {
        "print": {
          "histo": {
            "n_bins": 60,
            "fmt": {
              "svg": true
            }
          }
        }
      }
    }
  }
}
```

* or you can get the actual associations (either inner or left join)

```bash
{
  "qlist": {
    "right": "I/355/gaiadr3",
    "d_max": 3.0,
    "matches": {
      "all": {
        "limit": 100,
        "print": {
          "inner": {
            "print_coo": false
          }
        }
      }
    }
  }
}
```

```bash
{
  "qlist": {
    "right": "I/355/gaiadr3",
    "d_max": 3.0,
    "matches": {
      "all": {
        "limit": 100,
        "print": {
          "left": {
            "print_coo": false
          }
        }
      }
    }
  }
}
```

### Example of data file

The data file must be in CSV format.
The first column must be 64 bit unsigned integer identifer.
The second column must be the right ascension, in decimal degrees.
The third and last column must be the declination, also in decimal degrees.

```bash
> cat xmatch_data.csv
117204406,011.969660997391,-04.247611045837
117204407,011.972789049149,-04.246319055557
117204411,011.902252912521,-04.239807963371
117204412,011.896096944809,-04.243826985359
117204413,011.891215085983,-04.236031055450
117204413,011.891215085983,-04.236031055450
117204414,011.952013015747,-04.240033984184
117204415,011.939563035965,-04.234827995300
117204416,011.931442975998,-04.221395015717
...
```

### Limitations

The uploaded data (JSON + CSV) must be smaller than 5 MB.
This typically allows around 100,000 rows or more, depending on the identifier range
and the number of significant digits in the positions.

In addition:
* `dmax` must be lower (or equal to) 120 arcsec (i.e. 2 arcmin)
* the number of right entries returned by a left entries is limited to 100

### Example of xmatch submission using `curl`

```bash
  curl -X POST https://vizcat.cds.unistra.fr/cgi-bin/lbtree.cgi \
    -F "args=@xmatc_args.json;type=application/json" \
    -F "data=@xmatch_data.csv;type=application/octet-stream"
```