The EU Safety Gate Data - Feature Engineering

Basics

Duplicate Rows - There are no duplicated Rows

Strip - Strip leading / trailing spaces from df

Data Cleaning

Clean up mistyped accent characters, inconsistent punctuation, rename single use of country names, rename long country names to shorter versions to improve visualisation readability.

Feature Engineer - Date Time

Feature Engineer new columns from datetime index : Year, Week,

Feature Engineer - Economic Area

Fix Brand name / Country of origin inconsistencies

It's worthwhile to improve the quality of relevant data where possible for larger product categories.

Begin with

• 638 Brands
• 949 Brand / Country Combinations

After mapping Brands

• 428 Brands
• 718 Brand / Country Combinations

After mapping Countries

• 428 Brands
• 428 Brand / Country Combinations

Create consistent risk_category feature from Risk type column

Before

There are 218 variations in the `Risk type` column

Each entry is a comma separated list of risks.

The first entry in a list is chosen as the main category, i.e `Burns, Chemical, Damage to hearing` is mapped to `Burns`

A Future task is to split the comma separated variables into binary columns

After Feature Engineering

After Risk type field is feature engineered to risk_category, the result is 14 risk categories

Feature Engineer Category to product_category

There are two product category fields:

OECD portal category : there are 33 categories, however there are a lot of missing values

Category is complete with no missing values and has 80 categories.

The top 12 entries in category represent > 90% of the total number of alerts issued.
product_category will detail the top12 + others listed as other.

Before Feature Engineering

• 81 unique values

After Feature Engineering

• 13 product_category values (12 x named + other)

After mapping:

Feature Engineer country_top15

There are 111 countries of origin, the top 15 of these represent > 90% of the total alerts. Countries outside of the top 15 were aggregated under 'other'.

Binary Encode Compulsory measures / Voluntary measures columns

The Compulsory measures / Voluntary measures columns have detailed not delimited data, sometimes with multiple entries, indicating an escalation in measures.

To simplify the analysis and encoding, the encoding was matched in order of severity.

(I'd make a recommendation to the EU team to re-analyse the requirements of the database and improve this area of their record keeping.)

e.g.

Retailer: Withdrawal of the product from the market:27/05/2020 Retailer:Recall of the product from end users:27/05/2020 Distributor:Ban on the marketing of the product and any accompanying measures:27/05/2020

This example would be split into {compulsory/voluntary}{responsible_party} {action required}

e.g.

{compulsory/voluntary responsible party} options : unknown, retailer, distributor, importer, manufacturer, other

{compulsory/voluntary action required} options : Ban, recall, withdrawal, destruction, removal, criminal, borderstop, warnings

New columns with binary mappings added:

{compulsory_xxxx} or {voluntary_xxxxx} mapping

Compulsory measures mapping order of priority

Compulsory measures -> Binary columns encoding

Set binary values in column if true else false

Since multiple values can exist in each field, the order is relevant, we want to indicate severity and the simple contains search will match any instance, therefore, most import is the last search since it will overwrite previous matches

Result of mappings action required

Result of mappings responsible party

Creative Commons License

The reuse policy of European Commission documents is implemented by the Commission Decision 2011/833/EU of 12 December 2011 on the reuse of Commission documents (OJ L 330, 14.12.2011, p. 39). Except otherwise noted, the reuse of this document is authorised under a Creative Commons Attribution 4.0 International (CC-BY 4.0) licence.

This means that reuse is allowed provided appropriate credit is given and any changes are indicated.

Since this entire analysis is based on data from the EU, and I have used visualisations from the official annual summary as inspiration for the programmatically created version of these visualisations, the creative commons license applies this work hosted on rodders.me, this means that reuse of this analysis and its visualisations is allowed provided appropriate credit is given and any changes are indicated.