{"id":15,"date":"2024-04-04T00:39:46","date_gmt":"2024-04-03T21:39:46","guid":{"rendered":"https:\/\/sisu.ut.ee\/lcms_method_validation\/15-uploading-data-valchrom\/"},"modified":"2024-04-04T00:41:47","modified_gmt":"2024-04-03T21:41:47","slug":"15-uploading-data-valchrom","status":"publish","type":"page","link":"https:\/\/sisu.ut.ee\/lcms_method_validation\/15-uploading-data-valchrom\/","title":{"rendered":"1.5. Uploading data in ValChrom"},"content":{"rendered":"<p>\n\tIn this section we will upload our first dataset into ValChrom. Data can be uploaded in two main ways:\n<\/p>\n<ul>\n<li>\n\t\tCopy from a file and paste into ValChrom\n\t<\/li>\n<li>\n\t\tUploading a file (.xlsx, .csv, etc.)\n\t<\/li>\n<\/ul>\n<p>\n\tIn this MOOC you can use ready-made files for the problems presented in ValChrom-based self tests. Links to these files can be found within each such test.\u00a0\n<\/p>\n<h2>\n\tExperimental Datasets<br>\n<\/h2>\n<p>\n\tExperimental Datasets are used for holding results of experiments, which were carried out according to the Experimental Plan. The user can start populating a dataset when experiments for some validation parameter have been carried out, e.g. one can start assessing linearity when respective measurements are completed without the need to wait for all the rest of data to become available.\u00a0\n<\/p>\n<ol>\n<li>\n\t\tSections &gt; Experimental Datasets.\n\t<\/li>\n<li>\n\t\tClick on the large + sign at the right bottom of the window. \u00a0<img loading=\"lazy\" decoding=\"async\" width=\"89\" height=\"87\" class=\"alignnone wp-image-117\" style=\"width: 31px;height: 30px\" src=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/3_0_new_method_button.png\" title=\"_3_0_new_method_button.png\" alt=\"+\">\n\t<\/li>\n<li>\n\t\tInsert Dataset name: <strong>MOOC dataset<\/strong>\n\t<\/li>\n<li>\n\t\tSelect an Experimental Plan: <strong>MOOC experimental plan<\/strong><br>(If not available, then check that Experiment plan status is \u201cCOMPLETE\u201d)\n\t<\/li>\n<li>\n\t\tClick \u201cNext\u201d. You will be directed to upload view, where you can choose under which specific assessment method you want to upload data.\n\t<\/li>\n<\/ol>\n<p>\n\tData must to be uploaded for each assessment method separately. Click on \u201cClick to upload\u201d button or drag the file onto the button. Next we\u2019ll follow this with example data.\n<\/p>\n<p>\n\t<img loading=\"lazy\" decoding=\"async\" width=\"1481\" height=\"869\" class=\"alignnone wp-image-144\" src=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_1_new_upload_view.png\" title=\"1_5_1_new_upload_view.png\" alt=\"1.5.1\" srcset=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_1_new_upload_view.png 1481w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_1_new_upload_view-300x176.png 300w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_1_new_upload_view-1024x601.png 1024w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_1_new_upload_view-768x451.png 768w\" sizes=\"auto, (max-width: 1481px) 100vw, 1481px\">Choose assessment method \u201cBased on calibration graph | Slope and standard deviation of residuals\u201d by clicking \u201cUpload data\u201d button. You will be directed to data upload view.\n<\/p>\n<p>\n\t<img loading=\"lazy\" decoding=\"async\" width=\"1452\" height=\"681\" class=\"alignnone wp-image-153\" src=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_2_upload_data_01.png\" title=\"1_5_2_upload_data_01.png\" alt=\"upload_data\" srcset=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_2_upload_data_01.png 1452w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_2_upload_data_01-300x141.png 300w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_2_upload_data_01-1024x480.png 1024w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_2_upload_data_01-768x360.png 768w\" sizes=\"auto, (max-width: 1452px) 100vw, 1452px\">\n<\/p>\n<p>\n\tNext we will go over the two ways to upload data.\n<\/p>\n<h2>\n\tUploading data by copying from file\u00a0<br>\n<\/h2>\n<p>\n\tIn order to copy data:\n<\/p>\n<ol>\n<li>\n\t\tDownload the \u201ctest_data.xls\u201d file with raw data: <a data-fid=\"57390\" href=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/test_data.xls\">test_data.xls<\/a> and open the file.\n\t<\/li>\n<li>\n\t\tCopy the data from the file.\n\t<\/li>\n<\/ol>\n<p style=\"text-align: center\">\n\t<img loading=\"lazy\" decoding=\"async\" width=\"590\" height=\"248\" class=\"alignnone wp-image-146\" src=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_3_copy_1_file_01.png\" title=\"1_5_3_copy_1_file_01.png\" alt=\"1.5.3\" srcset=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_3_copy_1_file_01.png 590w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_3_copy_1_file_01-300x126.png 300w\" sizes=\"auto, (max-width: 590px) 100vw, 590px\">\n<\/p>\n<p>\n\t\u00a0\n<\/p>\n<p>\n\t3. Click on the field with \u201cClick on the box, then Ctrl+V or \u201cPaste\u201d\u201d, so that a cursor appears there.\n<\/p>\n<p>\n\t<img loading=\"lazy\" decoding=\"async\" width=\"752\" height=\"75\" class=\"alignnone wp-image-154\" src=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_4_copy_2_cursor_01_01.png\" title=\"1_5_4_copy_2_cursor_01_01.png\" alt=\"paste\" srcset=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_4_copy_2_cursor_01_01.png 752w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_4_copy_2_cursor_01_01-300x30.png 300w\" sizes=\"auto, (max-width: 752px) 100vw, 752px\">\n<\/p>\n<p>\n\t4. Paste your data (Ctrl + V).<br>5. You will see your data appear below at \u201cDataset preview\u201d. Here a quick glance at the data should be made in order to fix obvious errors immediately.\n<\/p>\n<p>\n\t<img loading=\"lazy\" decoding=\"async\" width=\"1107\" height=\"771\" class=\"alignnone wp-image-155\" src=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_5_copy_3_dataset_preview_01_01.png\" title=\"1_5_5_copy_3_dataset_preview_01_01.png\" alt=\"dataset upload\" srcset=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_5_copy_3_dataset_preview_01_01.png 1107w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_5_copy_3_dataset_preview_01_01-300x209.png 300w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_5_copy_3_dataset_preview_01_01-1024x713.png 1024w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_5_copy_3_dataset_preview_01_01-768x535.png 768w\" sizes=\"auto, (max-width: 1107px) 100vw, 1107px\">\n<\/p>\n<p>\n\t6. Click \u201cCheck data to proceed\u201d to check if all data is present.\u00a0<br>7. Also, a button \u201cShow results\u201d appears. Clicking this button will direct straight to the calculated results (see section \u201cOverview\u201d below). Clicking \u201cBack\u201d takes you back to \u201cUpload Experimental Dataset\u201d page.\n<\/p>\n<p style=\"text-align: center\">\n\t<img loading=\"lazy\" decoding=\"async\" width=\"266\" height=\"109\" class=\"alignnone wp-image-149\" src=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_6_copy_4_valid_01.png\" title=\"1_5_6_copy_4_valid_01.png\" alt=\"1.5.6\">\n<\/p>\n<h2>\n\tUploading a file\u00a0<br>\n<\/h2>\n<p>\n\tIn order to upload a file:<br>1. Download the \u201ctest_data.xls\u201d file with raw data: <a data-fid=\"57390\" href=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/test_data.xls\">test_data.xls<\/a>.<br>2. Click on the \u201cClick to upload\u201d button to find the file with the data.<br>3. Alternatively, you can drag and drop the file on the button.\n<\/p>\n<p>\n\t<img loading=\"lazy\" decoding=\"async\" width=\"707\" height=\"207\" class=\"alignnone wp-image-156\" src=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_7_upload_file_1_drag_01_01.png\" title=\"1_5_7_upload_file_1_drag_01_01.png\" alt=\"dataset upload\" srcset=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_7_upload_file_1_drag_01_01.png 707w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_7_upload_file_1_drag_01_01-300x88.png 300w\" sizes=\"auto, (max-width: 707px) 100vw, 707px\">\n<\/p>\n<p>\n\t4. From here on follow point 5. and onward in previous section \u201cUploading data by copying from file\u201d.\n<\/p>\n<h2>\n\tOverview\u00a0<br>\n<\/h2>\n<p>\n\t<strong>Overview <\/strong>is divided by validation parameter. Every validation parameter is presented in two sections: Overview and Detailed view. At once results for one analyte\u00a0are shown. Clicking on a specific validation parameter will open Overview for it.\u00a0\n<\/p>\n<p>\n\tIn Overview only very few data are shown, generally representative of all series or the outcome of the assessment method. For example, in case of LoD and LoQ this would be the overall LoD and LoQ values of the assessment method (taken as the maximum (conservative) values, if there are several series to choose from). In case of Accuracy, overview would have pooled standard deviation calculated over all series data. <strong>Detailed View<\/strong> offers all data available: input data and calculated results with graphs, if applicable.\n<\/p>\n<p>\n\t1. Go to Overview either by clicking \u201cShow Results\u201d button after uploading the data.<br>2. Alternatively, you can go to Sections &gt; Experimental Dataset. Find your dataset and click on Overview button or the Dataset name.\n<\/p>\n<p style=\"text-align: center\">\n\t<img loading=\"lazy\" decoding=\"async\" width=\"246\" height=\"159\" class=\"alignnone wp-image-136\" src=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/5_4_dataset_ov_button.png\" title=\"_5_4_dataset_ov_button.png\" alt=\"Sections\">\n<\/p>\n<p>\n\t3. You will see overview divided into several sections based on validation parameters. Clicking on a validation parameter will show Overview for assessment methods.\n<\/p>\n<p>\n\t<img loading=\"lazy\" decoding=\"async\" width=\"1101\" height=\"1008\" class=\"alignnone wp-image-151\" src=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_8_ov_generalview.png\" title=\"1_5_8_ov_generalview.png\" alt=\"1.5.8\" srcset=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_8_ov_generalview.png 1101w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_8_ov_generalview-300x275.png 300w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_8_ov_generalview-1024x938.png 1024w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_8_ov_generalview-768x703.png 768w\" sizes=\"auto, (max-width: 1101px) 100vw, 1101px\">\n<\/p>\n<p>\n\t4.\u00a0 Click on \u201cDetailed View\u201d.\u00a0\n<\/p>\n<p>\n\t<img loading=\"lazy\" decoding=\"async\" width=\"918\" height=\"1051\" class=\"alignnone wp-image-152\" src=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_9_ov_detailedview.png\" title=\"1_5_9_ov_detailedview.png\" alt=\"1.5.9\" srcset=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_9_ov_detailedview.png 918w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_9_ov_detailedview-262x300.png 262w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_9_ov_detailedview-894x1024.png 894w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/1_5_9_ov_detailedview-768x879.png 768w\" sizes=\"auto, (max-width: 918px) 100vw, 918px\">\n<\/p>\n<p>\n\t<br>5.\u00a0<strong>You can now see the automatically calculated results for the test data! <\/strong>In the \u201cInput data\u201d table the raw data used for the calculations can be seen, together with some calculated results for each table row (e.g. residuals). In \u201cSeries overall\u201d table data calculated from the whole series is shown (e.g. LoD and LoQ values, or standard deviation of residuals: SD(residuals)). Try clicking on \u201cLinear regression graph\u201d or \u201cResiduals graph\u201d.\n<\/p>\n<p>\n\t6. <strong>Note on MS Excel and programming languages precision<\/strong>: Excel is following double-precision floating-point format from the IEEE 754 specification. Strictly following the IEEE 754 specification causes some loss of precision, as can be seen from Microsoft\u00b4s documentation on the topic [ref: <a data-url=\"https:\/\/docs.microsoft.com\/en-us\/office\/troubleshoot\/excel\/floating-point-arithmetic-inaccurate-result\" href=\"https:\/\/docs.microsoft.com\/en-us\/office\/troubleshoot\/excel\/floating-point-arithmetic-inaccurate-result\" target=\"_blank\" title=\"\" rel=\"noopener\">https:\/\/docs.microsoft.com\/en-us\/office\/troubleshoot\/excel\/floating-point-arithmetic-inaccurate-result<\/a>]. Excel (and other spreadsheets) only store first 15 significant digits and all others are lost. In most practical circumstances this is not noticed nor relevant. However, in cases where raw data has many significant digits this imprecision will carry over to calculated results, like in case of large areas and unrounded theoretical concentrations with a lot of significant digits (e.g. 0.173240255623636). You can test this with the following data in ValChrom (or R\/Python\/Java):\u00a0<a data-fid=\"57407\" href=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/excel_vs_python_vs_java_vs_r_rss.xlsx\">Excel_vs_Python_vs_Java_vs_R_RSS.xslx<\/a>\n<\/p>\n<p>\n\tIn most usual cases the amount of significant digits does not get large enough and\/or the differences can mostly be observed in the 8th and beyond significant digit of the answer, so that very rarely in case of large unrounded numbers (slopes, RSS, etc.) can these differences be seen. In the examples of this MOOC the data provided is not meeting any of the extreme criteria, thus Excel and ValChrom will give the same answers.\n<\/p>\n<p>\n\t<strong>NB! <\/strong>Later in the course you\u2019ll upload data files under the same assessment methods \u2013 ValChrom will overwrite previously uploaded data. Use action button Upload under datasets table for this purpose.\n<\/p>\n<p style=\"text-align: center\">\n\t<img loading=\"lazy\" decoding=\"async\" width=\"401\" height=\"251\" class=\"alignnone wp-image-139\" style=\"width: 320px;height: 200px\" src=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/5_7_dataset_upload_button.png\" title=\"_5_7_dataset_upload_button.png\" alt=\"Upload\" srcset=\"https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/5_7_dataset_upload_button.png 401w, https:\/\/sisu.ut.ee\/wp-content\/uploads\/sites\/130\/5_7_dataset_upload_button-300x188.png 300w\" sizes=\"auto, (max-width: 401px) 100vw, 401px\">\n<\/p>\n<p>\n\t<strong>Well done! <\/strong>This concludes our introduction into uploading data in ValChrom.\u00a0<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this section we will upload our first dataset into ValChrom. Data can be uploaded in two main ways: Copy from a file and paste into ValChrom Uploading a file (.xlsx, .csv, etc.) In this MOOC you can use ready-made &#8230;<\/p>\n","protected":false},"author":60,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"class_list":["post-15","page","type-page","status-publish","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/sisu.ut.ee\/lcms_method_validation\/wp-json\/wp\/v2\/pages\/15","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sisu.ut.ee\/lcms_method_validation\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/sisu.ut.ee\/lcms_method_validation\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/sisu.ut.ee\/lcms_method_validation\/wp-json\/wp\/v2\/users\/60"}],"replies":[{"embeddable":true,"href":"https:\/\/sisu.ut.ee\/lcms_method_validation\/wp-json\/wp\/v2\/comments?post=15"}],"version-history":[{"count":1,"href":"https:\/\/sisu.ut.ee\/lcms_method_validation\/wp-json\/wp\/v2\/pages\/15\/revisions"}],"predecessor-version":[{"id":869,"href":"https:\/\/sisu.ut.ee\/lcms_method_validation\/wp-json\/wp\/v2\/pages\/15\/revisions\/869"}],"wp:attachment":[{"href":"https:\/\/sisu.ut.ee\/lcms_method_validation\/wp-json\/wp\/v2\/media?parent=15"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}