{"id":22545,"date":"2020-12-23T17:07:22","date_gmt":"2020-12-23T16:07:22","guid":{"rendered":"https:\/\/www.regestaitalia.eu\/machine-learning-quality-of-iot-data-as-a-success-factor\/"},"modified":"2025-06-03T15:33:31","modified_gmt":"2025-06-03T13:33:31","slug":"machine-learning-quality-of-iot-data-as-a-success-factor","status":"publish","type":"post","link":"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/","title":{"rendered":"Machine Learning: quality of IoT data as a success factor"},"content":{"rendered":"

In the article on <\/span>data valorization<\/a> we embarked on a brief journey to discover the declinations of the techniques of <\/span>Machine Learning<\/strong> combined with Internet of Things technologies.<\/span><\/p>\n

The applications that arise from this combination are certainly the most interesting and advanced within the Industry 4.0 world: also growing in interest in the operational and economic implications.<\/p>\n

Continuing down the path, after talking about <\/span>how to prepare data for Machine Learning we began to understand how carrying out extensive IoT data collection alone does not guarantee the success of the predictive project one wishes to undertake: central is the ability to extrapolate the crucial moments and aspects of the process being monitored.<\/span><\/p>\n

Last but not least.<\/span><\/em>, to extrapolate knowledge from past experience, it is necessary for the information collected to really tell the story of what happened: as in any project where data must turn into value, the theme of <\/span>data quality<\/strong> cannot be overlooked.<\/span><\/p>\n<\/div>

IoT data cleaning: what is it?<\/h2><\/div><\/div><\/div>

The <\/span>Data Cleaning<\/strong> (also referred to as Data Cleansing) is as much mentioned as it is vast to address: why is it so important?<\/span><\/p>\n

As in previous articles, we will try to understand what are the main critical issues that can occur when first approaching a<\/span> large amount of IoT data.<\/span><\/b>, to understand when Data Cleansing can intervene and what situations it can solve.<\/span><\/p>\n<\/div><\/div><\/div>

<\/span><\/div><\/div><\/div>

During all the time the sensors have been in collection from the field, events of any kind may have occurred and introduced some defection into the memories of the plants:<\/p>\n

    \n
  • The <\/span>time series of a monitored variable<\/strong> being monitored may contain a missing or significantly out-of-range value, leading to a <\/span>blackout <\/span><\/em>in the complete trend of the phenomenon;<\/span><\/li>\n
  • Some characteristics of the sensor, internal or external, may have varied over time. If the monitoring system is not properly hardened to these changes,  <\/span>one compromises the unambiguity of values sampled<\/strong> at different times in the face of the physical phenomenon.<\/span><\/li>\n<\/ul>\n

    These two simple examples identify as many seemingly similar but profoundly different situations regarding the possibility of intervention through <\/span>data cleaning techniques<\/strong>.<\/span><\/p>\n

    The case histories described in the first example do not create great concern within the proper functioning of the apparatus: the key is to be able to identify them and <\/span>apply the appropriate techniques to reconstruct the complete history<\/strong>.<\/span><\/p>\n

    The second example draws attention to the inescapable need for good health of the information-producing apparatus, on which its reliability depends.<\/p>\n

    If this pillar gives way, statistically any inference from any analysis can no longer be justified.<\/p>\n<\/div>

    Examples of Data Cleaning Approaches on IoT Data<\/h2><\/div>

    In some cases, the situation is easily manageable: think of a slow variable, as temperature trends often can be: even in the presence of a few <\/span>blackout <\/span><\/em>in sampling, we are able to estimate the average trend with good accuracy.<\/span><\/p>\n

    If, on the other hand, we were interested in <\/span>reconstructing the continuity of the original historical series<\/strong>, a <\/span>correct interpolation would be sufficient<\/strong> to obtain a sequence faithful to reality.<\/span><\/p>\n

    Other situations are more delicate, and a more focused approach is needed: think of the course of a current during a transient phase, where instantaneous values such as spikes and derivatives often encapsulate great indications of possible process anomalies.<\/p>\n<\/div>

    The Data Lake IoT Cloud as a Strategic Choice<\/h2><\/div>

    As seen above, there are some situations where no <\/span>data cleaning technique<\/span> <\/b>can reconstruct a consistent sequence of values within the collected data.<\/span><\/p>\n

    That is why the first fundamental condition for the successful application of Machine Learning algorithms lies in the<\/span> quality of the digitization process <\/strong>of the production line.<\/span><\/p>\n

    Today, a strategic choice is certainly to undertake the<\/span> building a data lake<\/span> <\/b>according to the most advanced methodologies and leveraging the right platforms, so that as soon as possible we can start harnessing the power of data<\/span><\/p>\n

    The use of platforms provided by major Cloud players (SAP, Google, AWS) can both ensure universally recognized solution quality and be the fastest way to achieve an<\/span> efficient data lake<\/strong> and reliable.<\/span><\/p>\n<\/div><\/div><\/div><\/div><\/div>\n","protected":false},"excerpt":{"rendered":"

    The quality of IoT data is definitely a factor in success-in fact, it is critical that the information collected really tells the story of what happened.<\/p>\n","protected":false},"author":1,"featured_media":21068,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[138],"tags":[],"class_list":["post-22545","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-iot-ml"],"yoast_head":"\nMachine Learning: data quality for success - Regesta LAB<\/title>\n<meta name=\"description\" content=\"The quality of IoT data is definitely a factor in success-in fact, it is critical that the information collected really tells the story of what happened.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Machine Learning: data quality for success - Regesta LAB\" \/>\n<meta property=\"og:description\" content=\"The quality of IoT data is definitely a factor in success-in fact, it is critical that the information collected really tells the story of what happened.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/\" \/>\n<meta property=\"og:site_name\" content=\"Regesta Group\" \/>\n<meta property=\"article:published_time\" content=\"2020-12-23T16:07:22+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-06-03T13:33:31+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.regestaitalia.eu\/wp-content\/uploads\/2020\/12\/regestaLab_20-12_ped_post10.jpg\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"9 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/\"},\"author\":{\"name\":\"admin\",\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/#\/schema\/person\/b2adaa1f97afe72b72049c658948c02b\"},\"headline\":\"Machine Learning: quality of IoT data as a success factor\",\"datePublished\":\"2020-12-23T16:07:22+00:00\",\"dateModified\":\"2025-06-03T13:33:31+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/\"},\"wordCount\":1784,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.regestaitalia.eu\/wp-content\/uploads\/2020\/12\/regestaLab_20-12_ped-1920_post10.jpg\",\"articleSection\":[\"Iot & ML\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/\",\"url\":\"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/\",\"name\":\"Machine Learning: data quality for success - Regesta LAB\",\"isPartOf\":{\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.regestaitalia.eu\/wp-content\/uploads\/2020\/12\/regestaLab_20-12_ped-1920_post10.jpg\",\"datePublished\":\"2020-12-23T16:07:22+00:00\",\"dateModified\":\"2025-06-03T13:33:31+00:00\",\"description\":\"The quality of IoT data is definitely a factor in success-in fact, it is critical that the information collected really tells the story of what happened.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/#primaryimage\",\"url\":\"https:\/\/www.regestaitalia.eu\/wp-content\/uploads\/2020\/12\/regestaLab_20-12_ped-1920_post10.jpg\",\"contentUrl\":\"https:\/\/www.regestaitalia.eu\/wp-content\/uploads\/2020\/12\/regestaLab_20-12_ped-1920_post10.jpg\",\"width\":1920,\"height\":1080},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.regestaitalia.eu\/en\/home-regesta-group\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Machine Learning: quality of IoT data as a success factor\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/#website\",\"url\":\"https:\/\/www.regestaitalia.eu\/en\/\",\"name\":\"Regesta Group\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.regestaitalia.eu\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/#organization\",\"name\":\"Regesta Group\",\"url\":\"https:\/\/www.regestaitalia.eu\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.regestaitalia.eu\/wp-content\/uploads\/2025\/04\/logo-per-yoast-100.jpg\",\"contentUrl\":\"https:\/\/www.regestaitalia.eu\/wp-content\/uploads\/2025\/04\/logo-per-yoast-100.jpg\",\"width\":697,\"height\":697,\"caption\":\"Regesta Group\"},\"image\":{\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/#\/schema\/person\/b2adaa1f97afe72b72049c658948c02b\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.regestaitalia.eu\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/cf4c27079a7ac863ff756baa403d32555186c1fdeaf53d13bd7cdde0966e8465?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/cf4c27079a7ac863ff756baa403d32555186c1fdeaf53d13bd7cdde0966e8465?s=96&d=mm&r=g\",\"caption\":\"admin\"},\"sameAs\":[\"https:\/\/regestaitalia.it\"],\"url\":\"https:\/\/www.regestaitalia.eu\/en\/author\/info_wrdz53pm\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Machine Learning: data quality for success - Regesta LAB","description":"The quality of IoT data is definitely a factor in success-in fact, it is critical that the information collected really tells the story of what happened.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/","og_locale":"en_US","og_type":"article","og_title":"Machine Learning: data quality for success - Regesta LAB","og_description":"The quality of IoT data is definitely a factor in success-in fact, it is critical that the information collected really tells the story of what happened.","og_url":"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/","og_site_name":"Regesta Group","article_published_time":"2020-12-23T16:07:22+00:00","article_modified_time":"2025-06-03T13:33:31+00:00","og_image":[{"url":"https:\/\/www.regestaitalia.eu\/wp-content\/uploads\/2020\/12\/regestaLab_20-12_ped_post10.jpg","type":"","width":"","height":""}],"author":"admin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"admin","Est. reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/#article","isPartOf":{"@id":"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/"},"author":{"name":"admin","@id":"https:\/\/www.regestaitalia.eu\/en\/#\/schema\/person\/b2adaa1f97afe72b72049c658948c02b"},"headline":"Machine Learning: quality of IoT data as a success factor","datePublished":"2020-12-23T16:07:22+00:00","dateModified":"2025-06-03T13:33:31+00:00","mainEntityOfPage":{"@id":"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/"},"wordCount":1784,"commentCount":0,"publisher":{"@id":"https:\/\/www.regestaitalia.eu\/en\/#organization"},"image":{"@id":"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/#primaryimage"},"thumbnailUrl":"https:\/\/www.regestaitalia.eu\/wp-content\/uploads\/2020\/12\/regestaLab_20-12_ped-1920_post10.jpg","articleSection":["Iot & ML"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/","url":"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/","name":"Machine Learning: data quality for success - Regesta LAB","isPartOf":{"@id":"https:\/\/www.regestaitalia.eu\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/#primaryimage"},"image":{"@id":"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/#primaryimage"},"thumbnailUrl":"https:\/\/www.regestaitalia.eu\/wp-content\/uploads\/2020\/12\/regestaLab_20-12_ped-1920_post10.jpg","datePublished":"2020-12-23T16:07:22+00:00","dateModified":"2025-06-03T13:33:31+00:00","description":"The quality of IoT data is definitely a factor in success-in fact, it is critical that the information collected really tells the story of what happened.","breadcrumb":{"@id":"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/#primaryimage","url":"https:\/\/www.regestaitalia.eu\/wp-content\/uploads\/2020\/12\/regestaLab_20-12_ped-1920_post10.jpg","contentUrl":"https:\/\/www.regestaitalia.eu\/wp-content\/uploads\/2020\/12\/regestaLab_20-12_ped-1920_post10.jpg","width":1920,"height":1080},{"@type":"BreadcrumbList","@id":"https:\/\/www.regestaitalia.eu\/en\/machine-learning-quality-of-iot-data-as-a-success-factor\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.regestaitalia.eu\/en\/home-regesta-group\/"},{"@type":"ListItem","position":2,"name":"Machine Learning: quality of IoT data as a success factor"}]},{"@type":"WebSite","@id":"https:\/\/www.regestaitalia.eu\/en\/#website","url":"https:\/\/www.regestaitalia.eu\/en\/","name":"Regesta Group","description":"","publisher":{"@id":"https:\/\/www.regestaitalia.eu\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.regestaitalia.eu\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.regestaitalia.eu\/en\/#organization","name":"Regesta Group","url":"https:\/\/www.regestaitalia.eu\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.regestaitalia.eu\/en\/#\/schema\/logo\/image\/","url":"https:\/\/www.regestaitalia.eu\/wp-content\/uploads\/2025\/04\/logo-per-yoast-100.jpg","contentUrl":"https:\/\/www.regestaitalia.eu\/wp-content\/uploads\/2025\/04\/logo-per-yoast-100.jpg","width":697,"height":697,"caption":"Regesta Group"},"image":{"@id":"https:\/\/www.regestaitalia.eu\/en\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.regestaitalia.eu\/en\/#\/schema\/person\/b2adaa1f97afe72b72049c658948c02b","name":"admin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.regestaitalia.eu\/en\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/cf4c27079a7ac863ff756baa403d32555186c1fdeaf53d13bd7cdde0966e8465?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/cf4c27079a7ac863ff756baa403d32555186c1fdeaf53d13bd7cdde0966e8465?s=96&d=mm&r=g","caption":"admin"},"sameAs":["https:\/\/regestaitalia.it"],"url":"https:\/\/www.regestaitalia.eu\/en\/author\/info_wrdz53pm\/"}]}},"_links":{"self":[{"href":"https:\/\/www.regestaitalia.eu\/en\/wp-json\/wp\/v2\/posts\/22545","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.regestaitalia.eu\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.regestaitalia.eu\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.regestaitalia.eu\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.regestaitalia.eu\/en\/wp-json\/wp\/v2\/comments?post=22545"}],"version-history":[{"count":1,"href":"https:\/\/www.regestaitalia.eu\/en\/wp-json\/wp\/v2\/posts\/22545\/revisions"}],"predecessor-version":[{"id":22550,"href":"https:\/\/www.regestaitalia.eu\/en\/wp-json\/wp\/v2\/posts\/22545\/revisions\/22550"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.regestaitalia.eu\/en\/wp-json\/wp\/v2\/media\/21068"}],"wp:attachment":[{"href":"https:\/\/www.regestaitalia.eu\/en\/wp-json\/wp\/v2\/media?parent=22545"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.regestaitalia.eu\/en\/wp-json\/wp\/v2\/categories?post=22545"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.regestaitalia.eu\/en\/wp-json\/wp\/v2\/tags?post=22545"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}