{ "cells": [ { "cell_type": "markdown", "source": [ "Validación con datos generados sintéticamente\n", "=============================================\n", "\n", "
PRECAUCIÓN 😱: El tema presentado en esta sección está clasificado como avanzado. El entendimiento de este contenido es totalmente opcional.
\n", "\n" ], "metadata": { "id": "ko9DnAUnfPdj" } }, { "cell_type": "markdown", "source": [ "No siempre es sencillo tener acceso a repositorios de datos que sean lo suficientemente grandes y con valores realistas (como ser nombre, edad, dirección, fecha de nacimiento, sexo, etc). Disponer de estos conjuntos de datos es útil no solo para aquellas personas que desean probar diferentes técnicas o modelos de aprendizaje automático, sino que también son muy interesantes para validadores de software (testers) que les gustaría tener un conjuntos de datos arbitrariamente grande (aunque con datos ficticios) sobre los cuales ejecutar pruebas de calidad de los modelos de aprendizaje que otros desarrolladores crearon. Esto facilita la posibilidad de probar determinados casos de borde sin necesidad de modificar datos manualmente ni tener que crear de forma artesanal conjuntos de datos para las validaciones." ], "metadata": { "id": "AxFpukOeh_ed" } }, { "cell_type": "markdown", "source": [ "## Sobre los datos sintéticos\n", "\n", "Los conjuntos de datos sintéticos son conjuntos de datos generados a traves de procesos artificiales con el proposito de mantener la privacidad de los datos originales, realizar validaciones de un sistema o pruebas de carga o simplemente verificar un modelo de aprendizaje automático. En general, los datos sintéticos se generan de tal forma que \"imitan\" la distribución o representación de un conjunto de datos real, pero sin necesidad de revelar ningún detalle del conjunto de datos original. En algunos casos puede ser que los datos generados sintéticamente no tengan el mismo valor que los datos reales, pero existen muchos otros casos donde pueden tener incluso el mismo valor. Muchos modelos de aprendizaje automático son entrenados con datos generados sintéticamente.\n", "\n", "Las generación de datos sintéticos se utiliza incluso durante el proceso de entrenamiento de modelos, con técnicas como `SMOTE` que permiten correjir el balance de clases en conjuntos de datos desbalanceados. Sin embargo, en esta sección no nos centraremos en tales típos de técnicas." ], "metadata": { "id": "jbch3-M2fPds" } }, { "cell_type": "markdown", "source": [ "## Técnicas para la generación de datos sintéticos\n", "\n", "Existen multiples técnicas para la generación de datos sintéticos dependiendo de los requerimientos de negocio y de las características de los conjuntos de datos que estamos buscando.\n", "\n", "### Generarlos desde distribuciones conocidas\n", "\n", "Aunque en la práctica no es el caso más habitual, puede ser que conoscamos la distribución de los datos que necesitamos, como ser una distribución normal, exponencial, binomial, etc. En tales casos, muchas librerías de aprendizaje automático nos permiten generar conjuntos de datos de tamaños arbitrales que se condicen con estas distribuciones.\n", "\n", "### Utilizando librerías espécificas\n", "\n", "Existen librerías específicas para generar conjuntos de datos sintéticos en múltiples lenguajes. En `Python`, librerías como las siguientes pueden ser utilizadas:\n", "\n", " - **faker**: Es una librería que no solo permite la generación de tipos de datos simples, sino que también datos como nombres, direcciones, teléfonos o correos electrónicos.\n", " - **SymPy:** Es una libería de alta performance que permite generar datos a traves de expresiones simbólicas.\n", " - **PyDBGen:** Es una librería que permite generar datos categóricos localizados (localized) para diferentes paises e idiomas. Puede generar conjuntos de datos tanto en archivos como csv o txt como directamente en tablas de bases de datos.\n", " - **mimesis:** Es una librería similar a faker, aunque provee multiples formas de extender sus funcionalidades.\n", "\n", "### Utilizando modelos generativos\n", "\n", "Modelos generativos, como son `Variational Autoencoders (VAE)` o `Generative Adversarial Networks (GAN)` puede generar datos sintéticos con un nivel de detallé importante. Muchas técnicas de aumento de datos (data augmentation) descansan en este tipo de modelos." ], "metadata": { "id": "iI8FYXEJfPdt" } }, { "cell_type": "markdown", "source": [ "## Ejemplos" ], "metadata": { "id": "h66mNxo-fPdu" } }, { "cell_type": "markdown", "source": [ "### Mimesis\n", "\n", "`Mimesis` es una libería de Python que permite generar conjuntos de datos sintéticos. `Mimesis` no solo permite la generación de tipos de datos simples, sino que también puede generar datos como nombres, direcciones, teléfonos o direcciones de correo electrónico. Esta información la puede generar para diferentes `locations` incluyendo el idioma Español. Lamentablemente, no dispone de una implementación específica para Argentina, siendo las variaciones más aproximadas las de Mexico (`es-mx`) o España (`es`). `Mimesis` es extensible, lo que significa que si existe un tipo de dato que no es soportado actualmente, se lo puede implementar para que la librería lo genere. Es ampliamente utilizado por la comunida de código abierto por lo que existen multiples extensiones ya creadas.\n", "\n", "Para más información sobre `Mimesis` puede [revisar la documentación oficial](https://mimesis.name/getting_started.html)." ], "metadata": { "id": "WhZ2lk2HfPdv" } }, { "cell_type": "code", "source": [ "%pip install mimesis==18.0.0" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "TZSHIDftFg8t", "outputId": "2975c017-4eeb-46cb-f792-eca4bef5f5ba" }, "execution_count": 23, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Requirement already satisfied: mimesis==18.0.0 in /usr/local/lib/python3.11/dist-packages (18.0.0)\n" ] } ] }, { "cell_type": "code", "execution_count": 10, "source": [ "import mimesis\n", "import mimesis.schema\n", "\n", "person = mimesis.Person(locale=\"es\")\n", "numbers = mimesis.Numeric()" ], "outputs": [], "metadata": { "id": "oshs2P57fPdw" } }, { "cell_type": "markdown", "source": [ "Por ejemplo, si quisieramos generar un nombre de una persona generalmente asociado a sexo femenino, podríamos hacer los siguiente:" ], "metadata": { "id": "Z7cwTsldfPdy" } }, { "cell_type": "code", "execution_count": null, "source": [ "print(person.full_name(gender=mimesis.enums.Gender.FEMALE))" ], "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Nieves Molina\n" ] } ], "metadata": { "id": "8s4Yg1mtfPdz", "outputId": "e9aaff65-8032-4777-99f1-8e55b5331d03" } }, { "cell_type": "markdown", "source": [ "De forma similar, podriamos generar la edad de una persona mayor a 21 años como sigue:" ], "metadata": { "id": "aipWhENjfPd2" } }, { "cell_type": "code", "execution_count": 11, "source": [ "print(numbers.integer_number(start=21, end=100))" ], "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "38\n" ] } ], "metadata": { "id": "3x9mex7ofPd4", "outputId": "1875276c-7f8e-4abe-9628-cd959dc472ef", "colab": { "base_uri": "https://localhost:8080/" } } }, { "cell_type": "markdown", "source": [ "Esta libreria nos permite generar conjuntos completos de datos de forma rápida indicando un **esquema de datos**. En el siguiente ejemplo, crearemos un conjunto de datos con varios campos, para luego construir un `DataFrame` de `Pandas`:" ], "metadata": { "id": "75K7WjngfPd5" } }, { "cell_type": "code", "execution_count": 20, "source": [ "field = mimesis.schema.Field(locale=\"es\")\n", "description = (\n", " lambda: {\n", " 'id': field('uuid'),\n", " 'creator': field('full_name', gender=mimesis.enums.Gender.FEMALE),\n", " 'age': field('integer_number', start=21, end=100),\n", " 'timestamp': field('datetime.datetime'),\n", " 'word': field('text.word'),\n", " 'email': field('person.email', domains=['outlook.com', 'gmail.com', 'yahoo.com'], key=str.lower),\n", " }\n", ")\n", "\n", "sample_schema = mimesis.schema.Schema(schema=description)" ], "outputs": [], "metadata": { "id": "M1i2gF20fPd5" } }, { "cell_type": "markdown", "source": [ "Creamos un `Pandas.DataFrame` con 100 muestras:" ], "metadata": { "id": "fwT5QYTcfPd6" } }, { "cell_type": "code", "execution_count": 21, "source": [ "import pandas as pd\n", "\n", "sample_schema.iterations = 100\n", "\n", "pd.DataFrame(sample_schema.create())" ], "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ " id creator age \\\n", "0 33fb7ea0-5a17-433a-a0e3-fcf5cedd8542 Narcisa Medina 87 \n", "1 df6a5130-949a-48b5-86ac-4fcb1f438613 Rosalía Lopez 83 \n", "2 55d77079-0370-4831-861f-30345bd474d3 Jessica Moya 38 \n", "3 67596f2d-5640-4073-bb3b-c40b3a36a130 Amalia Suarez 32 \n", "4 2273bf27-27c4-48bf-9915-f34420279a3d Meagens Santos 86 \n", ".. ... ... ... \n", "95 339bdc41-0cc3-4598-bf46-57cdbb548732 Begoña Fuentes 86 \n", "96 adf7546c-759c-43c6-823b-24f1b9571d8d Patricia Roman 42 \n", "97 4294fdac-5faa-45a3-a18a-f6d85e8ded14 Froilana Mendez 25 \n", "98 c76a3f9c-51c0-4d13-99c7-ebe6b3dbebda Teresa Diaz 76 \n", "99 b5617d1e-b148-4fa0-8116-1d26f003e794 Noe Vidal 47 \n", "\n", " timestamp word email \n", "0 2025-11-09 05:55:49.599516 viendo watershed1805@yahoo.com \n", "1 2025-10-28 04:21:49.607131 rápido wife1990@outlook.com \n", "2 2025-12-18 21:34:31.034018 toques rings2053@yahoo.com \n", "3 2025-11-24 13:27:43.362711 medicina portugal1850@gmail.com \n", "4 2025-12-16 05:53:38.754638 damas rack2022@outlook.com \n", ".. ... ... ... \n", "95 2025-04-26 07:33:08.491777 respuesta declare2001@yahoo.com \n", "96 2025-04-22 03:30:03.276388 parque kernel2053@yahoo.com \n", "97 2025-03-10 03:57:08.649930 pantalones senate1969@outlook.com \n", "98 2025-07-28 17:15:43.511954 esperaba replace1830@yahoo.com \n", "99 2025-04-25 20:24:44.440473 apoyo determined1945@gmail.com \n", "\n", "[100 rows x 6 columns]" ], "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
idcreatoragetimestampwordemail
033fb7ea0-5a17-433a-a0e3-fcf5cedd8542Narcisa Medina872025-11-09 05:55:49.599516viendowatershed1805@yahoo.com
1df6a5130-949a-48b5-86ac-4fcb1f438613Rosalía Lopez832025-10-28 04:21:49.607131rápidowife1990@outlook.com
255d77079-0370-4831-861f-30345bd474d3Jessica Moya382025-12-18 21:34:31.034018toquesrings2053@yahoo.com
367596f2d-5640-4073-bb3b-c40b3a36a130Amalia Suarez322025-11-24 13:27:43.362711medicinaportugal1850@gmail.com
42273bf27-27c4-48bf-9915-f34420279a3dMeagens Santos862025-12-16 05:53:38.754638damasrack2022@outlook.com
.....................
95339bdc41-0cc3-4598-bf46-57cdbb548732Begoña Fuentes862025-04-26 07:33:08.491777respuestadeclare2001@yahoo.com
96adf7546c-759c-43c6-823b-24f1b9571d8dPatricia Roman422025-04-22 03:30:03.276388parquekernel2053@yahoo.com
974294fdac-5faa-45a3-a18a-f6d85e8ded14Froilana Mendez252025-03-10 03:57:08.649930pantalonessenate1969@outlook.com
98c76a3f9c-51c0-4d13-99c7-ebe6b3dbebdaTeresa Diaz762025-07-28 17:15:43.511954esperabareplace1830@yahoo.com
99b5617d1e-b148-4fa0-8116-1d26f003e794Noe Vidal472025-04-25 20:24:44.440473apoyodetermined1945@gmail.com
\n", "

100 rows × 6 columns

\n", "
\n", "
\n", "\n", "
\n", " \n", "\n", " \n", "\n", " \n", "
\n", "\n", "\n", "
\n", " \n", "\n", "\n", "\n", " \n", "
\n", "\n", "
\n", "
\n" ], "application/vnd.google.colaboratory.intrinsic+json": { "type": "dataframe", "summary": "{\n \"name\": \"pd\",\n \"rows\": 100,\n \"fields\": [\n {\n \"column\": \"id\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 100,\n \"samples\": [\n \"4d719f80-1201-4497-be77-bdd52e544f4c\",\n \"d67a5e00-9cf9-40e4-9978-de6dd0086e09\",\n \"19fc660d-933a-4ef5-b761-9735c8354d07\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"creator\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 100,\n \"samples\": [\n \"Luciana Ortiz\",\n \"Soledad Gil\",\n \"Piedad Moreno\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"age\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 22,\n \"min\": 21,\n \"max\": 100,\n \"num_unique_values\": 57,\n \"samples\": [\n 87,\n 56,\n 78\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"timestamp\",\n \"properties\": {\n \"dtype\": \"date\",\n \"min\": \"2025-01-02 00:10:00.688886\",\n \"max\": \"2025-12-28 01:33:25.969389\",\n \"num_unique_values\": 100,\n \"samples\": [\n \"2025-03-27 23:45:13.869743\",\n \"2025-12-28 01:33:25.969389\",\n \"2025-12-14 06:32:25.631934\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"word\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 97,\n \"samples\": [\n \"asiento\",\n \"termin\\u00f3\",\n \"parque\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"email\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 100,\n \"samples\": [\n \"grades2082@yahoo.com\",\n \"taken1909@yahoo.com\",\n \"highly2080@yahoo.com\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}" } }, "metadata": {}, "execution_count": 21 } ], "metadata": { "id": "5n4RN0oofPd6", "outputId": "5e8a828b-aca4-4968-b87a-b0448ab89ba1", "colab": { "base_uri": "https://localhost:8080/", "height": 423 } } }, { "cell_type": "markdown", "source": [ "### CTGAN\n", "\n", "CTGAN es un técnica que ha ganado popularidad ya que se basa en la arquitectura GAN para modelar datos tabulares y muestrear filas condicionalmente a partir de un modelo generado especificamente para crear datos sintéticos. Al momento, esta representa una de las técnicas más avanzadas para la generación de datos sintéticos.\n", "\n", "[Xu, Lei and Skoularidou, Maria and Cuesta-Infante, Alfredo and Veeramachaneni, Kalyan. Modeling Tabular data using Conditional GAN. Advances in Neural Information Processing Systems. 2019](https://arxiv.org/abs/1907.00503)\n", "\n", "Para demostrar como realizar validaciones utilizando conjunto de datos sintéticos, trabajaremos con el conjunto de datos de 'Pima Indians Diabetes Database' (https://www.openml.org/d/37), proveniente originalmente del National Institute of Diabetes and Digestive and Kidney Diseases. El objetivo del conjunto de datos es el de predecir si un paciente tiene o no diabetes, basado en ciertas mediciones que se incluyen. Este conjunto es interesante para esta demostración porque tiene varias restricciones en cuanto a como se recolectaron los datos. En particular, todos los pacientes son de sexo femenino, de al menos 21 años y de ascendencia india Prima." ], "metadata": { "id": "rhfw4mtTfPd7" } }, { "cell_type": "code", "source": [ "%pip install ctgan" ], "metadata": { "id": "nwGryNHyHSg2" }, "execution_count": null, "outputs": [] }, { "cell_type": "code", "execution_count": 24, "source": [ "import pandas as pd\n", "import sklearn.datasets as datasets\n", "\n", "data = datasets.fetch_openml(name='diabetes', version=1)\n", "df = pd.DataFrame(data['data'], columns=data['feature_names'])\n", "df['target'] = data['target'].astype('object')" ], "outputs": [], "metadata": { "id": "7V9Odk8MfPd7" } }, { "cell_type": "markdown", "source": [ "Veamos como luce el conjunto de datos" ], "metadata": { "id": "W3Ivh2htfPd8" } }, { "cell_type": "code", "execution_count": 25, "source": [ "df.head(10)" ], "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ " preg plas pres skin insu mass pedi age target\n", "0 6 148 72 35 0 33.6 0.627 50 tested_positive\n", "1 1 85 66 29 0 26.6 0.351 31 tested_negative\n", "2 8 183 64 0 0 23.3 0.672 32 tested_positive\n", "3 1 89 66 23 94 28.1 0.167 21 tested_negative\n", "4 0 137 40 35 168 43.1 2.288 33 tested_positive\n", "5 5 116 74 0 0 25.6 0.201 30 tested_negative\n", "6 3 78 50 32 88 31.0 0.248 26 tested_positive\n", "7 10 115 0 0 0 35.3 0.134 29 tested_negative\n", "8 2 197 70 45 543 30.5 0.158 53 tested_positive\n", "9 8 125 96 0 0 0.0 0.232 54 tested_positive" ], "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
pregplaspresskininsumasspediagetarget
061487235033.60.62750tested_positive
11856629026.60.35131tested_negative
28183640023.30.67232tested_positive
318966239428.10.16721tested_negative
40137403516843.12.28833tested_positive
55116740025.60.20130tested_negative
637850328831.00.24826tested_positive
71011500035.30.13429tested_negative
82197704554330.50.15853tested_positive
9812596000.00.23254tested_positive
\n", "
\n", "
\n", "\n", "
\n", " \n", "\n", " \n", "\n", " \n", "
\n", "\n", "\n", "
\n", " \n", "\n", "\n", "\n", " \n", "
\n", "\n", "
\n", "
\n" ], "application/vnd.google.colaboratory.intrinsic+json": { "type": "dataframe", "variable_name": "df", "summary": "{\n \"name\": \"df\",\n \"rows\": 768,\n \"fields\": [\n {\n \"column\": \"preg\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 3,\n \"min\": 0,\n \"max\": 17,\n \"num_unique_values\": 17,\n \"samples\": [\n 6,\n 1,\n 3\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"plas\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 31,\n \"min\": 0,\n \"max\": 199,\n \"num_unique_values\": 136,\n \"samples\": [\n 151,\n 101,\n 112\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"pres\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 19,\n \"min\": 0,\n \"max\": 122,\n \"num_unique_values\": 47,\n \"samples\": [\n 86,\n 46,\n 85\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"skin\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 15,\n \"min\": 0,\n \"max\": 99,\n \"num_unique_values\": 51,\n \"samples\": [\n 7,\n 12,\n 48\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"insu\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 115,\n \"min\": 0,\n \"max\": 846,\n \"num_unique_values\": 186,\n \"samples\": [\n 52,\n 41,\n 183\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"mass\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 7.8841603203754405,\n \"min\": 0.0,\n \"max\": 67.1,\n \"num_unique_values\": 248,\n \"samples\": [\n 19.9,\n 31.0,\n 38.1\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"pedi\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.33132859501277484,\n \"min\": 0.078,\n \"max\": 2.42,\n \"num_unique_values\": 517,\n \"samples\": [\n 1.731,\n 0.426,\n 0.138\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"age\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 11,\n \"min\": 21,\n \"max\": 81,\n \"num_unique_values\": 52,\n \"samples\": [\n 60,\n 47,\n 72\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"target\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"tested_negative\",\n \"tested_positive\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}" } }, "metadata": {}, "execution_count": 25 } ], "metadata": { "id": "2q7a87S_fPd8", "outputId": "e0d42dda-151f-495b-c928-cae9ae527cd6", "colab": { "base_uri": "https://localhost:8080/", "height": 363 } } }, { "cell_type": "markdown", "source": [ "Entrenemos un modelo basado en la arquitectura de CTGAN para generar sintéticamente instancias pertenecientes a este conjunto de datos. Preste especial atención a los valores especificados en la variable `discrete_columns` donde indicamos aquellas variables que son categoricas. Las restantes variables serán consideradas como continuas." ], "metadata": { "id": "T3gdAa-xfPd9" } }, { "cell_type": "code", "execution_count": 27, "source": [ "from ctgan import CTGAN\n", "\n", "discrete_columns = ['preg', 'age', 'target']\n", "\n", "ctgan = CTGAN(epochs=5)\n", "ctgan.fit(df, discrete_columns)" ], "outputs": [], "metadata": { "id": "KJNBnR5jfPd-" } }, { "cell_type": "code", "execution_count": 28, "source": [ "ctgan.sample(10)" ], "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ " preg plas pres skin insu mass pedi age target\n", "0 1 135 46 37 98 31.233694 2.772864 45 tested_positive\n", "1 11 180 40 3 482 24.726927 0.578021 25 tested_negative\n", "2 2 128 68 7 79 48.442813 0.135262 54 tested_negative\n", "3 4 132 81 37 68 32.928613 0.582547 70 tested_positive\n", "4 9 133 80 -7 489 34.210912 0.551240 59 tested_negative\n", "5 1 97 61 32 364 51.610659 0.807465 21 tested_positive\n", "6 13 151 84 3 111 45.342234 0.391592 34 tested_positive\n", "7 1 218 76 14 154 29.559014 0.173615 38 tested_negative\n", "8 1 161 25 4 572 34.164945 0.340733 27 tested_negative\n", "9 12 100 77 21 499 -20.287110 0.568273 33 tested_negative" ], "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
pregplaspresskininsumasspediagetarget
0113546379831.2336942.77286445tested_positive
11118040348224.7269270.57802125tested_negative
221286877948.4428130.13526254tested_negative
3413281376832.9286130.58254770tested_positive
4913380-748934.2109120.55124059tested_negative
5197613236451.6106590.80746521tested_positive
61315184311145.3422340.39159234tested_positive
71218761415429.5590140.17361538tested_negative
8116125457234.1649450.34073327tested_negative
9121007721499-20.2871100.56827333tested_negative
\n", "
\n", "
\n", "\n", "
\n", " \n", "\n", " \n", "\n", " \n", "
\n", "\n", "\n", "
\n", " \n", "\n", "\n", "\n", " \n", "
\n", "\n", "
\n", "
\n" ], "application/vnd.google.colaboratory.intrinsic+json": { "type": "dataframe", "summary": "{\n \"name\": \"ctgan\",\n \"rows\": 10,\n \"fields\": [\n {\n \"column\": \"preg\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 5,\n \"min\": 1,\n \"max\": 13,\n \"num_unique_values\": 7,\n \"samples\": [\n 1,\n 11,\n 13\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"plas\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 36,\n \"min\": 97,\n \"max\": 218,\n \"num_unique_values\": 10,\n \"samples\": [\n 161,\n 180,\n 97\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"pres\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 20,\n \"min\": 25,\n \"max\": 84,\n \"num_unique_values\": 10,\n \"samples\": [\n 25,\n 40,\n 61\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"skin\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 15,\n \"min\": -7,\n \"max\": 37,\n \"num_unique_values\": 8,\n \"samples\": [\n 3,\n 14,\n 37\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"insu\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 207,\n \"min\": 68,\n \"max\": 572,\n \"num_unique_values\": 10,\n \"samples\": [\n 572,\n 482,\n 364\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"mass\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 20.08391754232703,\n \"min\": -20.28711019252924,\n \"max\": 51.610659308417546,\n \"num_unique_values\": 10,\n \"samples\": [\n 34.164945008858716,\n 24.726927263227815,\n 51.610659308417546\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"pedi\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.7598024812939892,\n \"min\": 0.13526232389257503,\n \"max\": 2.772863766429486,\n \"num_unique_values\": 10,\n \"samples\": [\n 0.3407328948835757,\n 0.5780208632041969,\n 0.8074651005928872\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"age\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 16,\n \"min\": 21,\n \"max\": 70,\n \"num_unique_values\": 10,\n \"samples\": [\n 27,\n 25,\n 21\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"target\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"tested_negative\",\n \"tested_positive\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}" } }, "metadata": {}, "execution_count": 28 } ], "metadata": { "id": "8hQc10OlfPd-", "outputId": "c85d5bfe-d7c8-466b-c3d5-fe071ca276d0", "colab": { "base_uri": "https://localhost:8080/", "height": 363 } } }, { "cell_type": "markdown", "source": [ ".. note:: Note que variables como `age` y `preg` que representan la edad de la persona y la cantidad de veces que ha estado embarazada fueron configuradas como variables categoricas. Esto permite que nuestro modelo 1) no genere valores continuos, como ser 20.5 para la edad, y 2) que no utilice valores que se encuentren fuera de los valores que ya existen en el conjunto de datos. En algunos casos, sobre todo considerando escenarios de testing, este comportamiento podría no ser el deseado." ], "metadata": { "id": "IerX5sadfPd_" } } ], "metadata": { "interpreter": { "hash": "bea38c2984299ac640e8421861d34b2e05ee614f6236d2975c05eeb77366835f" }, "kernelspec": { "name": "python3", "display_name": "Python 3.8.5 64-bit ('base': conda)" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" }, "orig_nbformat": 4, "colab": { "provenance": [], "toc_visible": true } }, "nbformat": 4, "nbformat_minor": 0 }