ASCII text
Data formats
This document describes the various data formats that are used in the system.
Raw annotation data
The client sends raw data for image annotations in a JSON format which is a list of shapes. Each shape is a dictionary with the following keys:
type: The type of the shape which can be:bbox(bounding box, rectangle)polygonpolylinepoint
shape: The shape data. Its format depends on the shapetype:For
bboxit is a dictionary with keys x, y, w, h: ~json {"x": x, "y": y, "w": w, "h": h}~For
polygonandpolylineit is a list of points; each point is a dictionary with keys x and y: ~json [{"x": x1, "y": y1}, {"x": x2, "y": y2}, ...]~ The only difference betweenpolygonandpolylineis that the former is supposed to be closed so the last point is connected to the first one.For
pointit is a dictionary with keys x and y: ~json {"x": x, "y": y}~All coordinates are floating-point numbers in the range [0, 1] and relative to the image size, with the origin in the top-left corner.
object: The ID of the type of object (label) depicted in the shape. This ID is a human-readable string that must be registered in the system before being used on shapes.
The server sends the same data back to the client, to use to show the existing annotations for an image.
Example
[
{
"type": "bbox",
"shape": {"x": 0.1, "y": 0.1, "w": 0.5, "h": 0.5},
"object": "Cat (Felis catus)"
},
{
"type": "polygon",
"shape": [{"x": 0, "y": 0}, {"x": 1, "y": 0}, {"x": 0, "y": 1}],
"object": "Slice of pizza margherita"
},
{
"type": "point",
"shape": {"x": 0.5, "y": 0.5},
"object": "Cat (Felis catus) - left eye"
}
]
Query format
The query format is based on YAML and used to query for pictures in the system.
Example
# Restrictions for queried images
- want:
# This means that the image must contain both rules, so both a cat and a dog
- has_object: ["Cat (Felis catus)"]
- has_object: ["Dog (Canis lupus familiaris)"]
# Or we can put them in a list to mean that the image can contain any of the
# objects in the list
- has_object: ["Grass", "Flower"]
# So the image must contain a cat and a dog, as well as either grass or
# a flower
# The following rule restricts the images to those with a certain source,
# like a camera or a drawing; omitting this rule means that the images can
# be of any type
- nature: ["photo", "drawing"]
# The following rule restricts the images to those with a certain licence
- licence: ["CC-BY-1.0", "CC-BY-2.0", "CC-BY-3.0", "CC-BY-4.0", "CC0-1.0",
"Unlicense", "WTFPL", "MIT", "BSD-2-Clause", "BSD-3-Clause",
"Apache-2.0", "Informal-attribution", "Informal-do-anything",
"Public-domain-old", "Public-domain"]
# Prohibitions for queried images
- exclude:
# This means that the image must not contain any of the objects in the list
- has_object: ["Human"]
# This excludes images taken before the given date
- before_date: 2019-01-01
# This requires images to have a minimum resolution
- below_width: 800
- below_height: 600
# The object types to get - if omitted, all object types mentioned in the
# `want` section are returned
- object_types: ["Cat (Felis catus)", "Dog (Canis lupus familiaris)"]
# Pagination
- limit: 32
- offset: 0
# Sorting
- sort_by: "date-uploaded-recent"
# Format
- format: "jpg"
- max_resolution: [800, 800] # resizes
# In summary, we want the 32 most recent images that contain both a cat and
# a dog, either a grass or a flower, but not a human, taken after 2019-01-01,
# must be a photo or a drawing, must carry one of certain permissive licences
# and have a resolution of at least 800x600 pixels. We don't need the object
# data for the plants, only for the pets.
1
Data formats
2
============
3
This document describes the various data formats that are used in the system.
5
Raw annotation data
7
-------------------
8
The client sends raw data for image annotations in a JSON format which is a list
10
of shapes. Each shape is a dictionary with the following keys:
11
* `type`: The type of the shape which can be:
13
* `bbox` (bounding box, rectangle)
14
* `polygon`
15
* `polyline`
16
* `point`
17
* `shape`: The shape data. Its format depends on the shape `type`:
18
* For `bbox` it is a dictionary with keys x, y, w, h:
19
~~~json
20
{"x": x, "y": y, "w": w, "h": h}
21
~~~
22
* For `polygon` and `polyline` it is a list of points; each point is a
23
dictionary with keys x and y:
24
~~~json
25
[{"x": x1, "y": y1}, {"x": x2, "y": y2}, ...]
26
~~~
27
The only difference between `polygon` and `polyline` is that the former is
28
supposed to be closed so the last point is connected to the first one.
29
* For `point` it is a dictionary with keys x and y:
30
~~~json
31
{"x": x, "y": y}
32
~~~
33
* All coordinates are floating-point numbers in the range [0, 1] and relative
34
to the image size, with the origin in the top-left corner.
35
* `object`: The ID of the type of object (label) depicted in the shape. This ID
36
is a human-readable string that must be registered in the system before
37
being used on shapes.
38
The server sends the same data back to the client, to use to show the existing
40
annotations for an image.
41
### Example
43
~~~json
45
[
46
{
47
"type": "bbox",
48
"shape": {"x": 0.1, "y": 0.1, "w": 0.5, "h": 0.5},
49
"object": "Cat (Felis catus)"
50
},
51
{
52
"type": "polygon",
53
"shape": [{"x": 0, "y": 0}, {"x": 1, "y": 0}, {"x": 0, "y": 1}],
54
"object": "Slice of pizza margherita"
55
},
56
{
57
"type": "point",
58
"shape": {"x": 0.5, "y": 0.5},
59
"object": "Cat (Felis catus) - left eye"
60
}
61
]
62
~~~
63
Query format
65
------------
66
The query format is based on YAML and used to query for pictures in the system.
68
### Example
72
~~~yaml
73
# Restrictions for queried images
74
- want:
75
# This means that the image must contain both rules, so both a cat and a dog
76
- has_object: ["Cat (Felis catus)"]
77
- has_object: ["Dog (Canis lupus familiaris)"]
78
# Or we can put them in a list to mean that the image can contain any of the
79
# objects in the list
80
- has_object: ["Grass", "Flower"]
81
# So the image must contain a cat and a dog, as well as either grass or
82
# a flower
83
# The following rule restricts the images to those with a certain source,
84
# like a camera or a drawing; omitting this rule means that the images can
85
# be of any type
86
- nature: ["photo", "drawing"]
87
# The following rule restricts the images to those with a certain licence
88
- licence: ["CC-BY-1.0", "CC-BY-2.0", "CC-BY-3.0", "CC-BY-4.0", "CC0-1.0",
89
"Unlicense", "WTFPL", "MIT", "BSD-2-Clause", "BSD-3-Clause",
90
"Apache-2.0", "Informal-attribution", "Informal-do-anything",
91
"Public-domain-old", "Public-domain"]
92
# Prohibitions for queried images
93
- exclude:
94
# This means that the image must not contain any of the objects in the list
95
- has_object: ["Human"]
96
# This excludes images taken before the given date
97
- before_date: 2019-01-01
98
# This requires images to have a minimum resolution
99
- below_width: 800
100
- below_height: 600
101
# The object types to get - if omitted, all object types mentioned in the
102
# `want` section are returned
103
- object_types: ["Cat (Felis catus)", "Dog (Canis lupus familiaris)"]
104
# Pagination
105
- limit: 32
106
- offset: 0
107
# Sorting
108
- sort_by: "date-uploaded-recent"
109
# Format
110
- format: "jpg"
111
- max_resolution: [800, 800] # resizes
112
# In summary, we want the 32 most recent images that contain both a cat and
113
# a dog, either a grass or a flower, but not a human, taken after 2019-01-01,
114
# must be a photo or a drawing, must carry one of certain permissive licences
115
# and have a resolution of at least 800x600 pixels. We don't need the object
116
# data for the plants, only for the pets.
117
~~~
118