Python学习记录 使用tensorflow 2.8 完成猫狗识别 使用keras构建CNN神经网络

猫狗识别

项目数据分为标记数据和未标记数据

[En]

Project data is divided into tagged and untagged

带标签:25000张
不带标签:12500张

文章目录

数据分类处理

数据集可以从kaggle下载
https://www.kaggle.com/c/dogs-vs-cats
也可以从我网盘下载
链接:https://pan.baidu.com/s/1_A3H4nsL8xX4cmMEMtpACg?pwd=mpvi
提取码:mpvi
–来自百度网盘超级会员V7的分享
下载的数据存放在data文件夹下

创建用于创建文件夹的函数

[En]

Create a function for creating a folder


def mkdir(path):
    if not os.path.exists(path):
        os.makedirs(path)

runPath = os.getcwd()
data_dir =runPath + '/data/'
mkdir(data_dir)

Python学习记录 使用tensorflow 2.8 完成猫狗识别 使用keras构建CNN神经网络
  • train是25000张带标签的猫狗图片
  • test1是12500张无标签的猫狗图片

我们将这25000张带标签的猫狗图片进行分类
分为训练集和验证集
训练集 20000张 验证集 5000张

猫的训练数据保存在 train/cat下 一共是10000张
猫的校验数据保存在 valid/cat下一共是2500张
狗的训练数据保存在 train/dog下 一共是10000张
狗的校验数据保存在 valid/dog下一共是2500张


train_dir = data_dir + 'train/'
mkdir(train_dir)

valid_dir = data_dir + 'valid/'
mkdir(valid_dir)

cat_dir = train_dir + 'cat/'
mkdir(cat_dir)

cat_val_dir = valid_dir + 'cat/'
mkdir(cat_val_dir)

dog_dir = train_dir + 'dog/'
mkdir(dog_dir)

dog_val_dir = valid_dir + 'dog/'
mkdir(dog_val_dir)

使用代码进行将这25000张带标签的猫狗图片进行分类


def move_file(file_path, target_path):

    fileName = os.path.split(file_path)[1]

    target_path+='/'+fileName
    print(target_path)
    if os.path.exists(file_path):
        shutil.move(file_path, target_path)
    else:
        print('文件不存在')

def changeFile(filetype):
    test_file = []
    for i in range(0,12500,1):
        filename = train_dir+filetype+"."+str(i)+'.jpg'
        print(filename)

        if i < 10000:
            move_file(filename,train_dir +filetype)

        else:
            move_file(filename,valid_dir +filetype)

changeFile('cat')

changeFile('dog')

Python学习记录 使用tensorflow 2.8 完成猫狗识别 使用keras构建CNN神经网络

检查分类情况

Python学习记录 使用tensorflow 2.8 完成猫狗识别 使用keras构建CNN神经网络
目录情况
Python学习记录 使用tensorflow 2.8 完成猫狗识别 使用keras构建CNN神经网络

图像增强预处理

训练前,我们将全部的图像输入ImageDataGenerator进行图像增强预处理
参数可以自行修改


def img_transforms():

    train_datagen = tf.keras.preprocessing.image.ImageDataGenerator(
            rescale=1. / 255,
            rotation_range=40,
            width_shift_range=0.2,
            height_shift_range=0.2,
            shear_range=0.2,
            zoom_range=0.2,
            horizontal_flip=True,
            fill_mode='nearest',
    )

    train_generator = train_datagen.flow_from_directory(
        train_dir,
        target_size=(128, 128),
        batch_size=64,
        seed=7,
        shuffle=True,
        class_mode='categorical'
    )

    valid_datagen = tf.keras.preprocessing.image.ImageDataGenerator(
        rescale=1. / 255,
    )

    valid_generator = valid_datagen.flow_from_directory(
        valid_dir,
        target_size=(128, 128),
        batch_size=64,
        seed=7,
        shuffle=False,
        class_mode="categorical"
    )
    return train_generator,valid_generator

train_generator,valid_generator = img_transforms()

编写神经网络结构

构建一个8层的神经网络,卷积层6层,全连接层2层,每两个卷积层添加一个池化层防止过拟合


def cnn(width,height,depth,outputNum):

    model = tf.keras.models.Sequential([

        tf.keras.layers.Conv2D(filters=32,
                            kernel_size=3,
                            padding='same',
                            activation='relu',
                            input_shape=[width, height, depth]),

        tf.keras.layers.Conv2D(filters=32,
                            kernel_size=3,
                            padding='same',
                            activation='relu'),

        tf.keras.layers.MaxPool2D(pool_size=2),

        tf.keras.layers.Conv2D(filters=64,
                            kernel_size=3,
                            padding='same',
                            activation='relu'),

        tf.keras.layers.Conv2D(filters=64,
                            kernel_size=3,
                            padding='same',
                            activation='relu'),

        tf.keras.layers.MaxPool2D(pool_size=2),

        tf.keras.layers.Conv2D(filters=128,
                            kernel_size=3,
                            padding='same',
                            activation='relu'),

        tf.keras.layers.Conv2D(filters=128,
                            kernel_size=3,
                            padding='same',
                            activation='relu'),

        tf.keras.layers.MaxPool2D(pool_size=2),

        tf.keras.layers.Flatten(),

        tf.keras.layers.Dense(128, activation='relu'),
        tf.keras.layers.Dense(outputNum, activation='softmax')
    ])

    model.compile(loss='categorical_crossentropy',
                  optimizer='adam',
                  metrics=['accuracy'])

    model.summary()
    return model

model = cnn(128,128,3,2)

设置模型保存路径

将模型保存在model文件夹下的catvsdog_weights.h5中


modelPath = './model'
mkdir(modelPath)

output_model_file = os.path.join(modelPath,"catvsdog_weights.h5")

输入样本进入模型进行训练

设置为回调模式,使用model.fit进行训练
使用EarlyStopping,当准确率没有提高超过一定幅度的话就会停止训练


def plot_learning_curves(history, label, epochs, min_value, max_value):
    data = {}
    data[label] = history.history[label]
    data['val_' + label] = history.history['val_' + label]
    pd.DataFrame(data).plot(figsize=(8, 5))
    plt.grid(True)
    plt.axis([0, epochs, min_value, max_value])
    plt.show()

TRAIN_STEP = 100

callbacks = [
            tf.keras.callbacks.TensorBoard(modelPath),
            tf.keras.callbacks.ModelCheckpoint(output_model_file,
                                            save_best_only=True,
                                            save_weights_only=True),
            tf.keras.callbacks.EarlyStopping(patience=5, min_delta=1e-3)
        ]

history = model.fit(
        train_generator,
        epochs=TRAIN_STEP,
        validation_data = valid_generator,
        callbacks = callbacks
    )

plot_learning_curves(history, 'accuracy', TRAIN_STEP, 0, 1)
plot_learning_curves(history, 'loss', TRAIN_STEP, 0, 5)

Python学习记录 使用tensorflow 2.8 完成猫狗识别 使用keras构建CNN神经网络

Python学习记录 使用tensorflow 2.8 完成猫狗识别 使用keras构建CNN神经网络
Python学习记录 使用tensorflow 2.8 完成猫狗识别 使用keras构建CNN神经网络

可以看到训练到40步的时候准确率已经变化不大了

加载模型进行预测

准备两张要预测的图片

Python学习记录 使用tensorflow 2.8 完成猫狗识别 使用keras构建CNN神经网络
dog.jpg
Python学习记录 使用tensorflow 2.8 完成猫狗识别 使用keras构建CNN神经网络
cat.jpg
也可以直接使用test1下的图片进行预测,这里只是觉得这两张图可爱

def predict(cvImg,model,modelPath,modelName,width,height,depth,result1,result2):
    model_file = os.path.join(modelPath,modelName)

    model.load_weights(modelName)

    img = cv2.resize(cvImg, (width, height))

    img_arr = img / 255.0

    img_arr = img_arr.reshape((1, width, height, depth))

    pre = model.predict(img_arr)

    if pre[0][0] > pre[0][1]:
        print("识别结果是:"+result1+"\n概率:"+str(pre[0][0]))
    else:
        print("识别结果是:"+result2+"\n概率:"+str(pre[0][1]))
img = cv2.imread("cat.jpg")
cv2.imshow("img",img)
predict(img,model,modelPath,"catvsdog_weights.h5",128,128,3,"猫","狗")
cv2.waitKey(0)

识别结果为猫

Python学习记录 使用tensorflow 2.8 完成猫狗识别 使用keras构建CNN神经网络
识别结果为狗
Python学习记录 使用tensorflow 2.8 完成猫狗识别 使用keras构建CNN神经网络

从测试集中随机选择一张图片进行预测

[En]

Randomly select a picture from the test set to predict


    test_dir = data_dir + "test1/"
    readImg = test_dir + str(random.randint(0,12499)) + ".jpg"
    img = cv2.imread(readImg)
    predict(img,model,modelPath,modelName,simpleWight,simpleHeight,simpleDepth,result1,result2)
    cv2.imshow("img",img)
    cv2.waitKey(0)

Python学习记录 使用tensorflow 2.8 完成猫狗识别 使用keras构建CNN神经网络

完整代码

import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'

CPU = 1
if CPU == 0 :
    os.environ['CUDA_VISIBLE_DEVICES'] = '-1'
else :
    os.environ['CUDA_VISIBLE_DEVICES'] = '0'

import random
import shutil
import cv2
import tensorflow as tf
import pandas as pd
import matplotlib.pyplot as plt

mode = 1

def move_file(file_path, target_path):

    fileName = os.path.split(file_path)[1]

    target_path+='/'+fileName
    print(target_path)
    if os.path.exists(file_path):
        shutil.move(file_path, target_path)
    else:
        print('文件不存在')

def mkdir(path):
    if not os.path.exists(path):
        os.makedirs(path)

def changeFile(filetype):
    test_file = []
    for i in range(0,12500,1):
        filename = train_dir+filetype+"."+str(i)+'.jpg'
        print(filename)

        if i < 10000:
            move_file(filename,train_dir +filetype)

        else:
            move_file(filename,valid_dir +filetype)

def img_transforms(train_dir,valid_dir):

    train_datagen = tf.keras.preprocessing.image.ImageDataGenerator(
            rescale=1. / 255,
            rotation_range=40,
            width_shift_range=0.2,
            height_shift_range=0.2,
            shear_range=0.2,
            zoom_range=0.2,
            horizontal_flip=True,
            fill_mode='nearest',
    )

    train_generator = train_datagen.flow_from_directory(
        train_dir,
        target_size=(128, 128),
        batch_size=64,
        seed=7,
        shuffle=True,
        class_mode='categorical'
    )

    valid_datagen = tf.keras.preprocessing.image.ImageDataGenerator(
        rescale=1. / 255,
    )

    valid_generator = valid_datagen.flow_from_directory(
        valid_dir,
        target_size=(128, 128),
        batch_size=64,
        seed=7,
        shuffle=False,
        class_mode="categorical"
    )
    return train_generator,valid_generator

def cnn(width,height,depth,outputNum):

    model = tf.keras.models.Sequential([

        tf.keras.layers.Conv2D(filters=32,
                            kernel_size=3,
                            padding='same',
                            activation='relu',
                            input_shape=[width, height, depth]),

        tf.keras.layers.Conv2D(filters=32,
                            kernel_size=3,
                            padding='same',
                            activation='relu'),

        tf.keras.layers.MaxPool2D(pool_size=2),

        tf.keras.layers.Conv2D(filters=64,
                            kernel_size=3,
                            padding='same',
                            activation='relu'),

        tf.keras.layers.Conv2D(filters=64,
                            kernel_size=3,
                            padding='same',
                            activation='relu'),

        tf.keras.layers.MaxPool2D(pool_size=2),

        tf.keras.layers.Conv2D(filters=128,
                            kernel_size=3,
                            padding='same',
                            activation='relu'),

        tf.keras.layers.Conv2D(filters=128,
                            kernel_size=3,
                            padding='same',
                            activation='relu'),

        tf.keras.layers.MaxPool2D(pool_size=2),

        tf.keras.layers.Flatten(),

        tf.keras.layers.Dense(128, activation='relu'),
        tf.keras.layers.Dense(outputNum, activation='softmax')
    ])

    model.compile(loss='categorical_crossentropy',
                  optimizer='adam',
                  metrics=['accuracy'])

    model.summary()
    return model

def train(model,modelPath,modelName,step,train_generator,valid_generator,isplot=False):

    def plot_learning_curves(history, label, epochs, min_value, max_value):
        data = {}
        data[label] = history.history[label]
        data['val_' + label] = history.history['val_' + label]
        pd.DataFrame(data).plot(figsize=(8, 5))
        plt.grid(True)
        plt.axis([0, epochs, min_value, max_value])
        plt.show()

    output_model_file = os.path.join(modelPath,modelName)

    callbacks = [
                tf.keras.callbacks.TensorBoard(modelPath),
                tf.keras.callbacks.ModelCheckpoint(output_model_file,
                                                save_best_only=True,
                                                save_weights_only=True),
                tf.keras.callbacks.EarlyStopping(patience=5, min_delta=1e-3)
            ]

    history = model.fit(
            train_generator,
            epochs=step,
            validation_data = valid_generator,
            callbacks = callbacks
        )
    if isplot:

        plot_learning_curves(history, 'accuracy', TRAIN_STEP, 0, 1)
        plot_learning_curves(history, 'loss', TRAIN_STEP, 0, 5)

def predict(cvImg,model,modelPath,modelName,width,height,depth,result1,result2):

    model_file = os.path.join(modelPath,modelName)

    model.load_weights(model_file)

    img = cv2.resize(cvImg, (width, height))

    img_arr = img / 255.0

    img_arr = img_arr.reshape((1, width, height, depth))

    pre = model.predict(img_arr)

    if pre[0][0] > pre[0][1]:
        print("识别结果是:"+result1+"\n概率:"+str(pre[0][0]))
    else:
        print("识别结果是:"+result2+"\n概率:"+str(pre[0][1]))

runPath = os.getcwd()
data_dir =runPath + '/data/'
mkdir(data_dir)

train_dir = data_dir + 'train/'
mkdir(train_dir)

valid_dir = data_dir + 'valid/'
mkdir(valid_dir)

cat_dir = train_dir + 'cat/'
mkdir(cat_dir)

cat_val_dir = valid_dir + 'cat/'
mkdir(cat_val_dir)

dog_dir = train_dir + 'dog/'
mkdir(dog_dir)

dog_val_dir = valid_dir + 'dog/'
mkdir(dog_val_dir)

modelPath = './model'
mkdir(modelPath)

modelName = "catvsdog_weights.h5"

simpleWight = 128
simpleHeight = 128
simpleDepth = 3

outputNum = 2
result1 = "猫"
result2 = "狗"

TRAIN_STEP = 100

model = cnn(simpleWight,simpleHeight,simpleDepth,outputNum)

if mode == 0:

    changeFile('cat')

    changeFile('dog')

    train_generator,valid_generator = img_transforms(train_dir,valid_dir)

    train(model,modelPath,modelName,TRAIN_STEP,train_generator,valid_generator)
elif mode == 1:

    test_dir = data_dir + "test1/"
    readImg = test_dir + str(random.randint(0,12499)) + ".jpg"
    img = cv2.imread(readImg)
    predict(img,model,modelPath,modelName,simpleWight,simpleHeight,simpleDepth,result1,result2)
    cv2.imshow("img",img)
    cv2.waitKey(0)

Original: https://blog.csdn.net/lx7820336/article/details/124768769
Author: 三千院喵
Title: Python学习记录 使用tensorflow 2.8 完成猫狗识别 使用keras构建CNN神经网络

原创文章受到原创版权保护。转载请注明出处:https://www.johngo689.com/496472/

转载文章受原作者版权保护。转载请注明原作者出处!

(0)

大家都在看

亲爱的 Coder【最近整理,可免费获取】👉 最新必读书单  | 👏 面试题下载  | 🌎 免费的AI知识星球