content/Multimodal-Dataset-Creation.md

@@ -4,6 +4,12 @@ The source code documentation is in the repository "multimodalDatasetBuilder/doc
 
 [[_TOC_]]
 
+## Introduction
+
+In the multimodal dataset creation, sentences of documents are enriched by images which in best case represent the context of these sentences. Such an image is called "main image". A multimodal sentence with a main image will also have at least one focus word. A focus word is defined as a word that is complex and depictable/concrete at the same time. The complex word identifier classifies if a word is complex. It can be turned off. Then every word is classified as complex. The depictability/concreteness property of a word is mainly derived from the concreteness values file. These concreteness values are calculated over the image dataset beforehand. For every focus word in a sentence, the main image of the sentence will be saved in a version in which the focus word is highlighted.
+
+At the end of the pipeline, every document with at least one multimodal sentence will be saved in a MongoDB database. The database contains two collections/tables. In the first collection, the documents are saved. For each sentence of a document, there is also an id (SHA-512) value of that sentence and the information if the sentence is multimodal. The second collection contains the multimodal sentences. For each sentence, the path to the main image is saved and a dictionary with the (multimodal) focus words and the path to their highlighted images. These informations can be accessed through the [API](./api).
+
 ## Required Packages
 
 Most of the required packages can be installed with pip using the "requirements.txt" file from the repository:
@@ -69,7 +75,7 @@ The following parameters are optional.
 * `--resized_main_images` - on default, the size of the main images will not change. If a directory is specified the main images of the sentences will be resized and saved there. The resized images will have the same size as the highlighted images which is 224x224 pixels
 * `-t, --concreteness_threshold` - defaults to 20. Filters out the words from the concreteness file which have a score lower than this threshold
 * `--cwi` - defaults to "on". The status that specifies if the cwi is "on" or "off"
-* `--multimodal_sents_only` - defaults to "off". The status that specifies if only the multimodal sentences of a documents will be saved
+* `--multimodal_sents_only` - defaults to "off". The status that specifies if only the multimodal sentences of the documents will be saved
 * `--max_docs` - defaults to None. Specifies the maximum number of documents that will be saved
 * `--max_sents` - defaults to None. Specifies the maximum number of sentences that a document is allowed to have to be further processed
 * `--rnd_seed` - defaults to None. Is responsible for the shuffling of the documents before they are processed