{"id":7294,"date":"2020-06-19T11:25:42","date_gmt":"2020-06-19T11:25:42","guid":{"rendered":"https:\/\/www.price2spy.com\/blog\/?p=7294"},"modified":"2020-07-28T10:49:04","modified_gmt":"2020-07-28T10:49:04","slug":"part-4-preparing-the-ml-training-set","status":"publish","type":"post","link":"https:\/\/www.price2spy.com\/blog\/part-4-preparing-the-ml-training-set\/","title":{"rendered":"(Part #4) Preparing the ML training set"},"content":{"rendered":"\n

Product matching in Price2Spy<\/a> <\/p>\n\n\n\n

Previous topic:<\/strong> (Part #3) For ML experts \u2013 why is product matching so difficult? <\/a> <\/p>\n\n\n\n

Next topic:<\/strong> (Part #5) ML training Implementation <\/a> <\/p>\n\n\n\n

This part was particularly tough. When performing the product matching<\/a>, one would need to match Set A (Products from Website A) to Set B (Products from Website B).<\/p>\n\n\n\n

\"ML<\/figure><\/div>\n\n\n\n

However, in order to avoid biased training, we needed to ensure to get as many different Sets A and Sets B \u2013 so from as many different websites as possible.<\/p>\n\n\n\n

Further, SetsA and Sets B need to be variable in size \u2013 we cannot expect all sites we will be matching to have a similar number of products (for example client\u2019s website might have 5000 products, and he may want it matched on Amazon, where this product category has more than 500 000 products).<\/p>\n\n\n\n

As for the language \u2013 in order to make our model language-independent, we were looking for a non-English language, but from an industry where product naming has many English influences.<\/p>\n\n\n\n

The choice of the industry was a bit easier. Going for an industry that has strong usage of MPNs in product names (like the automotive industry) would be too easy \u2013 the ML model would learn to rely on MPN too much, and that would not be applicable to other industries. And remember \u2013 our solution has to be industry-independent.<\/p>\n\n\n\n

Last but not least \u2013 anyone who\u2019s been dealing with ML knows – the size matters. The bigger your training set, the more reliable your ML model gets.<\/p>\n\n\n\n

But how do you prepare the training data for an ML model? That\u2019s the toughest part of all \u2013 someone (humans) needs to prepare accurate matches for all combinations of Sets A \/ Sets B.<\/p>\n\n\n\n

Luckily enough, Price2Spy<\/a> has been in business for long enough to acquire more than 650 clients from various parts of the world (various languages) and various industries. However, one industry is our sweet spot \u2013 and that is Music. Roughly 25% of our clients are from the musical world, and quite a few are from Germany. Musical gear has a great mixture of product naming variations, as you can see in examples below:<\/p>\n\n\n\n

1.HagstromFantomen Black<\/strong> \u2013 naming variations<\/p>\n\n\n\n