HDFS Write/read Speeds

Machine details - CPU/RAM

File read and write test Results Expertus network

Using cp command Local ⇒ Hdfs -

File size Machine Start time End time Remarks
Small file (mention size 50 MB) 10.229.94.18 Less than 1Sec
Medium file (mention size 615 Mb) 10.229.94.1806:02:43 06:03:44
Large file (mention size 1.1GB) 10.229.94.18 08:36:47 08:36:52
Very large file (mention size 2.4 GB) 10.229.94.18 05:14:4405:14:55

Using cp command Hdfs ⇒ Local -

File size Machine Start time End time Remarks
Small file (mention size 50 MB) 10.229.94.18 Less than 1Sec
Medium file (mention size 615 Mb)10.229.94.18 06:07:0106:07:03
Large file (mention size 1.1GB ) 10.229.94.18 08:38:13 08:38:18
Very large file (mention size 2.4 GB) 10.229.94.1805:18:30 05:19:35
File read and write test Results 192.168.1.10 Machine

Using cp command Local ⇒ Hdfs -

File size Machine Start time End time Remarks
Small file (mention size 50 MB) 10.229.94.18 Less than 1Sec
Medium file (mention size 615 Mb) 10.229.94.18 07:52:20 07:52:23
Large file (mention size 1.1GB) 10.229.94.18 07:55:18 07:55:24
Very large file (mention size 2.4 GB) 10.229.94.18 07:46:52 07:47:14

Using cp command Hdfs ⇒ Local -

File size Machine Start time End time Remarks
Small file (mention size 50 MB) 10.229.94.18 Less than 1Sec
Medium file (mention size 615 Mb)10.229.94.18 08:05:39 08:05:41
Large file (mention size 1.1GB ) 10.229.94.1808:02:52 08:02:56
Very large file (mention size 2.4 GB) 10.229.94.18 07:58:20 07:58:27

From the app through upload (this includes app execution time) -

File size Machine Read time Write time
Small file (mention size)
Medium file (mention size)
Large file (mention size)
Very large file(mention size)