Skip to content

Templates

create_templates(client)

Creates the data templates required for both latest and archive indices.

Parameters:

Name Type Description Default
client Elasticsearch

an Elasticsearch client object

 required
Source code in splitgill/indexing/templates.py 
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
def create_templates(client: Elasticsearch):
    """
    Creates the data templates required for both latest and archive indices.

    :param client: an Elasticsearch client object
    """
    client.indices.put_index_template(
        name='arc-data-template',
        body=get_arc_template(),
    )
    client.indices.put_index_template(
        name='latest-data-template',
        body=get_latest_template(),
    )

get_arc_template()

Returns the template to use for the archive (arc) indices. This template is the same as the base template but with a single shard per index as we limit the size of the index by record count, and therefore it's enough to just use a single shard.

Returns:

Type Description
dict

the template as a dict
Source code in splitgill/indexing/templates.py 
40
41
42
43
44
45
46
47
48
49
50
51
52
def get_arc_template() -> dict:
    """
    Returns the template to use for the archive (arc) indices. This template is the same
    as the base template but with a single shard per index as we limit the size of the
    index by record count, and therefore it's enough to just use a single shard.

    :return: the template as a dict
    """
    return _get_template(
        pattern='data-*-arc-*',
        shards=1,
        priority=700,
    )

get_latest_template()

Returns the template to use for the archive (arc) indices. This template is the same as the base template but with 5 shards per index to help search throughput and because the latest index could be large if the database has a large number of records.

Returns:

Type Description
dict

the template as a dict
Source code in splitgill/indexing/templates.py 
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
def get_latest_template() -> dict:
    """
    Returns the template to use for the archive (arc) indices. This template is the same
    as the base template but with 5 shards per index to help search throughput and
    because the latest index could be large if the database has a large number of
    records.

    :return: the template as a dict
    """
    return _get_template(
        pattern='data-*-latest',
        shards=5,
        # must be higher priority than the arc template otherwise the patterns can't be
        # resolved when they both match an index name
        priority=800,
    )