1. Let's take a look at the data structure of mnist first:

2.

As you can see, in train-images.idx3-ubyte, the first number is a 32-bit integer (magic number, picture type number), the second number is a 32-bit integer (picture number), the third and fourth numbers are also 32-bit integers (representing the number of rows and columns of pictures respectively), and the next number is an unsigned number of one byte (that is, pixels, ranging from 0 to 0).

3. How to parse data with Python? First of all, you need to install python graphics processing library PIL, which supports pixel-level image processing and is very helpful for learning digital image processing. After the installation is completed, the image can be analyzed. Look at the code:

4. Open the file first, and then read the magic number, picture number, row number and column number respectively. In struct, you can see' >; IIII, what does this mean? This means using the big-endian rule to read four integers. If you want to read a byte, you can use' >; B: Of course, it's the same rule whether to use big terminals here, so only two or more bytes are useful.

5. What is the big end rule? If you don't understand, you can Baidu, so I won't go into details (/link? URL = bgg 8 b 0 vrr 3 b _ segyol 8 u 4 dmabiqt 9 swguntd _ 2 1 ctei _ NliqsQ-mkf 73 yt 90 eilf 2 eqy 50 meua _ m 4 z 6 CMA 3 rmk)

6. Then, for each picture, first create a blank picture, where' L' means that this picture is a gray image, and finally read it pixel by pixel, then write it into the blank picture, and finally save the picture.

7. Let's take a look at the data structure of mnist tag:

You can see that it is very similar to the above, except that every byte here becomes a label (the label size is 0~9).

8. Well, through the above explanation, we can finally parse mnist through python, and see the effect:

The program source code is as follows:

#! /usr/bin/env python

#-*-encoding: utf-8 -*-

Import pictures from PIL

Import structure

Def read_image (file name):

F = open (file name, "rb")

Exponent = 0

buf = f.read()

f.close()

magic，images，rows，columns = struct . unpack _ from(' & gt； IIII '，buf，index)

index+= struct . calc size(' & gt； IIII’)

For i (image) in xrange:

# For I(2000) in xrange:

Image = Image.new('L', (column, row))

For x (line) in xrange:

For y (column) in xrange:

image.putpixel((y，x)，int(struct . unpack _ from(' & gt； b '，buf，index)[0]))

index+= struct . calc size(' & gt； b’)

Print "Save"+String (i)+"Image"

image.save('test/' + str(i) +')。 png’)

Def read_label (file name, save file name):

F = open (file name, "rb")

Exponent = 0

buf = f.read()

f.close()

magic，labels = struct . unpack _ from(' & gt； II', buffer, index)

index+= struct . calc size(' & gt； II')

LabelArr = [0] * label

#labelArr = [0] * 2000

For x (label) in xrange:

# For x(2000) in xrange:

label arr[x]= int(struct . unpack _ from(' & gt； b '，buf，index)[0])

index+= struct . calc size(' & gt； b’)

Save = open (save file name,' w')

save.write('，')。 join(map(lambda x: str(x)，labelArr)))

save.write('\n ')

save.close()

Print "label saved successfully"

if __name__ == '__main__ ':

read _ image(' t 10k-images . idx 3-ubyte ')

read _ label(' t 10k-labels . idx 1-ubyte '，' test/label.txt ')