Browse Source

updated readme

master
flashlan 3 years ago
parent
commit
13cdd81787
  1. 14
      Readme.md
  2. 7
      compareDBs.ipynb

14
Readme.md

@ -5,24 +5,16 @@
#### How to use: #### How to use:
* Rename the file "properties-model.ini" to "properties.ini" * Rename the file "properties-model.ini" to "properties.ini"
* Fill with your own credentials * Fill with your own credentials
* pip install requeriments.txt * `pip install requeriments.txt`
---- ----
The proposal of this work is to compare the speed in read/writing a midle level of data ( a dataset with 9 columns and 50.000 lines) to four diferent databases: The proposal of this work is to compare the speed in read/writing a midle level of data (a dataset with 9 columns and 1.000.000 lines) to seven diferent databases:
* ClickHouse * ClickHouse
* InfluxDB * InfluxDB
* Postgresql * Postgresql
* Parquet (in a S3 Minio Storage)<br> * Parquet (in a S3 Minio Storage)<br>
ToDo: * DuckDB
* DuckDB with Polars
* MongoDB * MongoDB
* Kdb+ * Kdb+
Deve-se relevar:
é uma "cold-storage" ou "frezze-storage"? <br>
influxdb: alta leitura e possui a vantagem da indexaçõa para posterior vizualização de dados em gráficos
notas:
* comparar tamanho do csv com parquet

7
compareDBs.ipynb

@ -1249,7 +1249,6 @@
"cell_type": "markdown", "cell_type": "markdown",
"id": "4409cc89-ed14-4313-ac89-65b826038533", "id": "4409cc89-ed14-4313-ac89-65b826038533",
"metadata": { "metadata": {
"jp-MarkdownHeadingCollapsed": true,
"tags": [] "tags": []
}, },
"source": [ "source": [
@ -1258,7 +1257,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null, "execution_count": 31,
"id": "14f63810-1943-4e28-9bce-2148be6be02d", "id": "14f63810-1943-4e28-9bce-2148be6be02d",
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
@ -1269,7 +1268,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null, "execution_count": 32,
"id": "8ff6c090-7e02-435a-a179-f2aab81da972", "id": "8ff6c090-7e02-435a-a179-f2aab81da972",
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
@ -1300,7 +1299,7 @@
"outputs": [], "outputs": [],
"source": [ "source": [
"%%time\n", "%%time\n",
"# send df to kd+ in memory bank\n", "# send df to kbd+ in memory bank\n",
"q.sendSync(\"{t::x}\", data)" "q.sendSync(\"{t::x}\", data)"
] ]
}, },

Loading…
Cancel
Save