Skip to content

Tools API Documentation

AudioPlugin

Convenience base class for plugins having audio as input type.

sr class-attribute

PluginParameter instance holding the sample rate at which the audio file must be loaded, the original sample rate is used if set to 0

__call__(self, inFilename, outFilename) special

Internal call of the plugin, loads the plugin if necessary, then loads the input audio file and calls the process method.

Parameters:

Name Type Description Default
inFilename str

The path of the file to be processed

required
outFilename str

The path to which the processed file must be written

required
Source code in dataset_explorer/plugins/base.py
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
def __call__(self, inFilename: str, outFilename: str):
    """
    Internal call of the plugin, loads the plugin if necessary, then loads the input audio file and calls the
    `process` method.

    Args:
        inFilename: The path of the file to be processed
        outFilename: The path to which the processed file must be written
    """
    if not self._loaded:
        self.load()
        self._loaded = True
    loadSr = self.sr.value if self.sr.value else None
    data, self.sr.value = librosa.load(inFilename, sr=loadSr)
    self.process(data, outFilename)

__init__(self, name, outType, icon='settings', outExtension='') special

Creates a AudioPlugin instance.

Parameters:

Name Type Description Default
name str

The name is the plugin as it will be displayed within the web interface

required
outType FileType

The output type of the plugin

required
icon Optional[str]

The name of the icon, must be one of the Angular Material Icon list

'settings'
outExtension Optional[str]

If specified, overrides the default file extension for the provided output file

''

Note

Default file extensions for each type are:

  • Image: .png
  • Audio: .wav
  • Video: .mp4
  • Text: .txt
  • Misc: no extension
Source code in dataset_explorer/plugins/base.py
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
def __init__(self, name: str, outType: FileType, icon: Optional[str] = 'settings', outExtension: Optional[str] = ''):
    """
    Creates a AudioPlugin instance.

    Args:
        name: The name is the plugin as it will be displayed within the web interface
        outType: The output type of the plugin
        icon: The name of the icon, must be one of the Angular Material Icon list
        outExtension: If specified, overrides the default file extension for the provided output file

    Note:
        Default file extensions for each type are:

        - Image: .png
        - Audio: .wav
        - Video: .mp4
        - Text: .txt
        - Misc: _no extension_
    """
    super(AudioPlugin, self).__init__(name, FileType.AUDIO, outType, icon, outExtension)

process(self, data, outFilename)

The main entrypoint of audio plugins, must be overriden in child class. The processed file must be saved at the path pointed by the argument outFilename.

Parameters:

Name Type Description Default
data ndarray

The audio signal as a numpy ndarray

required
outFilename str

The path to which the processed file must be written

required
Source code in dataset_explorer/plugins/base.py
205
206
207
208
209
210
211
212
213
214
def process(self, data: np.ndarray, outFilename: str):
    """
    The main entrypoint of audio plugins, must be overriden in child class.
    The processed file must be saved at the path pointed by the argument `outFilename`.

    Args:
        data: The audio signal as a numpy ndarray
        outFilename: The path to which the processed file must be written
    """
    raise NotImplementedError

BasePlugin

Base class for plugins.

__call__(self, inFilename, outFilename) special

Internal call of the plugin, loads the plugin if necessary and calls the process method.

Parameters:

Name Type Description Default
inFilename str

The path of the file to be processed

required
outFilename str

The path to which the processed file must be written

required
Source code in dataset_explorer/plugins/base.py
112
113
114
115
116
117
118
119
120
121
122
123
def __call__(self, inFilename: str, outFilename: str):
    """
    Internal call of the plugin, loads the plugin if necessary and calls the `process` method.

    Args:
        inFilename: The path of the file to be processed
        outFilename: The path to which the processed file must be written
    """
    if not self._loaded:
        self.load()
        self._loaded = True
    self.process(inFilename, outFilename)

__init__(self, name, inType, outType, icon='settings', outExtension='') special

Creates a BasePlugin instance.

Parameters:

Name Type Description Default
name str

The name is the plugin as it will be displayed within the web interface

required
inType FileType

The input type of the plugin

required
outType FileType

The output type of the plugin

required
icon Optional[str]

The name of the icon, must be one of the Angular Material Icon list

'settings'
outExtension Optional[str]

If specified, overrides the default file extension for the provided output file

''

Note

Default file extensions for each type are:

  • Image: .png
  • Audio: .wav
  • Video: .mp4
  • Text: .txt
  • Misc: no extension
Source code in dataset_explorer/plugins/base.py
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
def __init__(self, name: str, inType: FileType, outType: FileType, icon: Optional[str] = 'settings', outExtension: Optional[str] = ''):
    """
    Creates a BasePlugin instance.

    Args:
        name: The name is the plugin as it will be displayed within the web interface
        inType: The input type of the plugin
        outType: The output type of the plugin
        icon: The name of the icon, must be one of the Angular Material Icon list
        outExtension: If specified, overrides the default file extension for the provided output file

    Note:
        Default file extensions for each type are:

        - Image: .png
        - Audio: .wav
        - Video: .mp4
        - Text: .txt
        - Misc: _no extension_
    """
    self.name = name
    self.inType = inType
    self.outType = outType
    self.parameters = self._retrieveParameters(self.__class__, list())
    self.icon = icon
    if outExtension:
        self.outExtension = f".{outExtension.replace('.', '')}"
    else:
        self.outExtension = FileType.getDefaultExtension(self.outType)
    self._loaded = False

getFileHash(self, dataFile, params)

Computes and returns a unique hash for the three elements:

  • Plugin
  • Input data file
  • Parameters

Allows for caching already processed files.

Parameters:

Name Type Description Default
dataFile DataFile

The DataFile instance of the file to process

required
params dict

The user-specified parameters

required

Returns:

Type Description
int

A unique hash for the combination (Plugin + File + Params)

Source code in dataset_explorer/plugins/base.py
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
def getFileHash(self, dataFile: DataFile, params: dict) -> int:
    """
    Computes and returns a unique hash for the three elements:

    - Plugin
    - Input data file
    - Parameters

    Allows for caching already processed files.

    Args:
        dataFile: The DataFile instance of the file to process
        params: The user-specified parameters

    Returns:
        A unique hash for the combination (Plugin + File + Params)
    """
    parametersCopy = {name: parameter.value for name, parameter in self.parameters.items()}
    parametersCopy.update(params)
    parametersCopy["__datafile"] = dataFile.id
    parametersCopy["__plugin"] = self.name
    return hash(frozenset(parametersCopy.items()))

load(self)

Can be overriden in child class. Will be called automatically once before the first call to process is issued.

Source code in dataset_explorer/plugins/base.py
125
126
127
128
129
def load(self):
    """
    Can be overriden in child class. Will be called automatically once before the first call to `process` is issued.
    """
    pass

process(self, inFilename, outFilename)

The main entrypoint of plugins, must be overriden in child class. The processed file must be saved at the path pointed by the argument outFilename.

Parameters:

Name Type Description Default
inFilename str

The path of the file to be processed

required
outFilename str

The path to which the processed file must be written

required
Source code in dataset_explorer/plugins/base.py
131
132
133
134
135
136
137
138
139
140
def process(self, inFilename: str, outFilename: str):
    """
    The main entrypoint of plugins, must be overriden in child class.
    The processed file must be saved at the path pointed by the argument `outFilename`.

    Args:
        inFilename: The path of the file to be processed
        outFilename: The path to which the processed file must be written
    """
    raise NotImplementedError

setParameterValues(self, params)

Overrides the values of all the parameters of the plugin to match the user-specified params dict. If not specified, they are set to their default value.

Parameters:

Name Type Description Default
params dict

The user-specified params dict. The syntax of the latter is:

{
    "paramName1": value,
    "paramName2": value
}

required
Source code in dataset_explorer/plugins/base.py
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
def setParameterValues(self, params: dict):
    """
    Overrides the values of all the parameters of the plugin to match the user-specified params dict.
    If not specified, they are set to their default value.

    Args:
        params: The user-specified params dict.
            The syntax of the latter is:
            ```python
            {
                "paramName1": value,
                "paramName2": value
            }
            ```
    """
    for parameter in self.parameters.values():
        parameter.reset()
    for name, value in params.items():
        self.parameters[name].value = value

toJson(self)

Generates and returns the JSON representation of the plugin.

Returns:

Type Description
dict

The JSON representation of the plugin as a dict

Source code in dataset_explorer/plugins/base.py
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
def toJson(self) -> dict:
    """
    Generates and returns the JSON representation of the plugin.

    Returns:
        The JSON representation of the plugin as a dict
    """
    return {
        "className": self.__class__.__name__,
        "name": self.name,
        "inType": self.inType.value,
        "outType": self.outType.value,
        "parameters": [parameter.toJson(name) for name, parameter in self.parameters.items()],
        "icon": self.icon
    }

ImagePlugin

Convenience base class for plugins having image as input type.

__call__(self, inFilename, outFilename) special

Internal call of the plugin, loads the plugin if necessary, then loads the input image file and calls the process method.

Parameters:

Name Type Description Default
inFilename

The path of the file to be processed

required
outFilename

The path to which the processed file must be written

required
Source code in dataset_explorer/plugins/base.py
243
244
245
246
247
248
249
250
251
252
253
254
255
256
def __call__(self, inFilename, outFilename):
    """
    Internal call of the plugin, loads the plugin if necessary, then loads the input image file and calls the
    `process` method.

    Args:
        inFilename: The path of the file to be processed
        outFilename: The path to which the processed file must be written
    """
    if not self._loaded:
        self.load()
        self._loaded = True
    data = cv2.imread(inFilename)
    self.process(data, outFilename)

__init__(self, name, outType, icon='settings', outExtension='') special

Creates a ImagePlugin instance.

Parameters:

Name Type Description Default
name str

The name is the plugin as it will be displayed within the web interface

required
outType FileType

The output type of the plugin

required
icon Optional[str]

The name of the icon, must be one of the Angular Material Icon list

'settings'
outExtension Optional[str]

If specified, overrides the default file extension for the provided output file

''

Note

Default file extensions for each type are:

  • Image: .png
  • Audio: .wav
  • Video: .mp4
  • Text: .txt
  • Misc: no extension
Source code in dataset_explorer/plugins/base.py
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
def __init__(self, name: str, outType: FileType, icon: Optional[str] = 'settings', outExtension: Optional[str] = ''):
    """
    Creates a ImagePlugin instance.

    Args:
        name: The name is the plugin as it will be displayed within the web interface
        outType: The output type of the plugin
        icon: The name of the icon, must be one of the Angular Material Icon list
        outExtension: If specified, overrides the default file extension for the provided output file

    Note:
        Default file extensions for each type are:

        - Image: .png
        - Audio: .wav
        - Video: .mp4
        - Text: .txt
        - Misc: _no extension_
    """
    super(ImagePlugin, self).__init__(name, FileType.IMAGE, outType, icon, outExtension)

process(self, data, outFilename)

The main entrypoint of image plugins, must be overriden in child class. The processed file must be saved at the path pointed by the argument outFilename.

Parameters:

Name Type Description Default
data ndarray

The image as a numpy ndarray

required
outFilename

The path to which the processed file must be written

required
Source code in dataset_explorer/plugins/base.py
258
259
260
261
262
263
264
265
266
267
def process(self, data: np.ndarray, outFilename):
    """
    The main entrypoint of image plugins, must be overriden in child class.
    The processed file must be saved at the path pointed by the argument `outFilename`.

    Args:
        data: The image as a numpy ndarray
        outFilename: The path to which the processed file must be written
    """
    raise NotImplementedError

PluginParameter

User-editable parameter for plugins.

__init__(self, name, value) special

Create a PluginParameter instance.

Parameters:

Name Type Description Default
name str

The name of the parameter, it will be displayed within the web interface

required
value Union[int, float, str, bool]

The default value of the parameter

required
Source code in dataset_explorer/plugins/parameters.py
12
13
14
15
16
17
18
19
20
21
22
def __init__(self, name: str, value: Union[int, float, str, bool]):
    """
    Create a PluginParameter instance.

    Args:
        name: The name of the parameter, it will be displayed within the web interface
        value: The default value of the parameter
    """
    self.name = name
    self.value = value
    self._defaultValue = value

reset(self)

Sets back the value of the parameter to its default value.

Source code in dataset_explorer/plugins/parameters.py
24
25
26
27
28
def reset(self):
    """
    Sets back the value of the parameter to its default value.
    """
    self.value = self._defaultValue

toJson(self, attributeName)

Generates and returns the JSON representation of the parameter.

Parameters:

Name Type Description Default
attributeName str

The name of the Python instance

required

Returns:

Type Description
dict

The JSON representation of the parameter as a dict

Source code in dataset_explorer/plugins/parameters.py
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
def toJson(self, attributeName: str) -> dict:
    """
    Generates and returns the JSON representation of the parameter.

    Args:
        attributeName: The name of the Python instance

    Returns:
        The JSON representation of the parameter as a dict
    """
    return {
        "attributeName": attributeName,
        "name": self.name,
        "value": self._defaultValue,
        "default": self._defaultValue,
        "type": type(self.value).__name__
    }

InstantiationError

Raised when a plugins fails to instantiate

InvalidParameter

Raised when an invalid parameter is passed to a Plugin instance

OutputFileNotFound

Raised when the output file of a plugin does not exist

PluginError

Raised when a plugin-based exception occurs

ProcessError

Raised when a plugin failed to process some data