项目实战:Qt+OpenCV大家来找茬(Qt抓图,穿透应用,识别左右图区别,框选区别,微调位置)

若该文为原创文章,转载请注明出处
本文章博客地址https://hpzwl.blog.csdn.net/article/details/124768637
红胖子(红模仿)的博文大全:开发技术集合(包含Qt实用技术、树莓派、三维、OpenCV、OpenGL、ffmpeg、OSG、单片机、软硬结合等等)持续更新中…(点击传送门)

前言

本项目的出现理由只是笔者的一个念头,于是利用专业Qt和Opencv相关的知识开发一个辅助工具,本文章仅用于Qt和Opencv结合的学习。

Demo演示效果

项目实战:Qt+OpenCV大家来找茬(Qt抓图,穿透应用,识别左右图区别,框选区别,微调位置)
项目实战:Qt+OpenCV大家来找茬(Qt抓图,穿透应用,识别左右图区别,框选区别,微调位置)
项目实战:Qt+OpenCV大家来找茬(Qt抓图,穿透应用,识别左右图区别,框选区别,微调位置)
项目实战:Qt+OpenCV大家来找茬(Qt抓图,穿透应用,识别左右图区别,框选区别,微调位置)

; 运行包下载地址(供测试学习)

CSDN粉丝0积分下载地址:https://download.csdn.net/download/qq21497936/85372782
QQ群下载地址: 1047134658(点击” 文件“搜索” findTheDifference“,群内与博文同步更新)

运行包+源码包下载地址(供测试学习)

CSDN下载地址:https://download.csdn.net/download/qq21497936/85372767
(注意:源码本博客后面都有,若是想一步到位,下载这个,源码编译版本为Qt5.9.x mingw32 + openCV3.4.10)

功能列表

  • 应用程序可将某Q游戏界面套入内部区域,游戏方便操作;
  • 抓图区域调整,可通过右上角区域,调整区域1和区域2的位置;
  • 位置微调功能,点击按钮可像对应方向微调一个像素;
  • 识别不同,调用opencv算法,识别不同处在游戏图上绘制显示区域;
  • 游戏界面区域操作焦点为游戏界面;
  • 可清空已经绘制的区域;

Qt技术点

OpenCV技术点

项目模块化部署

项目的环境为Qt5.9.3 mingw32版本,使用QtCreator开发,配合mingw32版本的Opencv3.4.10,下图左侧为项目结构,右侧为实际文件夹部署结构。

项目实战:Qt+OpenCV大家来找茬(Qt抓图,穿透应用,识别左右图区别,框选区别,微调位置)

; Qt代码:DrawWdget

该类的主要作用:

  • 覆盖在游戏窗口上
  • 全部透明窗口用以当作游戏界面上的画布
  • 对鼠标消息穿透(无法点击中)
  • 识别出后绘制标记处不同的区域

Ui界面

为自动生成默认的,没有任何改动。

项目实战:Qt+OpenCV大家来找茬(Qt抓图,穿透应用,识别左右图区别,框选区别,微调位置)
一共绘制两类图形,一类是框出抓取图的界面,一类是识别出后的区域,定义两个缓存变量,用以绘制对应的区域矩形。

; DrawWidegt.h

#ifndef DRAWWIDGET_H
#define DRAWWIDGET_H

#include

namespace Ui {
class DrawWidget;
}

class DrawWidget : public QWidget
{
    Q_OBJECT

public:
    explicit DrawWidget(QWidget *parent = 0);
    ~DrawWidget();

public:
    void setRect(int x, int y, int width, int height);
    void setRect2(int x, int y, int width, int height);
    void clearListRect();
    void setListRect(QList<QRect> listRect);

protected:
    void initControl();

protected:
    void paintEvent(QPaintEvent *event);

protected:
    void drawSelectRect(QPainter *painter);
    void drawListRect(QPainter *painter);

private:
    Ui::DrawWidget *ui;

private:
    QColor _colorRect;
    QRect _rect;
    QRect _rect2;

    QList<QRect> _listRect;

};

#endif

DrawWidget.cpp

#include "DrawWidget.h"
#include "ui_DrawWidget.h"
#include
#include

#include
#include

#define LOG qDebug()<<__FILE__<<__LINE__<<QDateTime::currentDateTime().toString("yyyy-MM-dd hh:mm:ss:zzz")

DrawWidget::DrawWidget(QWidget *parent) :
    QWidget(parent),
    ui(new Ui::DrawWidget),
    _colorRect(Qt::red)
{
    ui->setupUi(this);

    setWindowFlag(Qt::FramelessWindowHint);
    setAttribute(Qt::WA_TranslucentBackground);
    setAttribute(Qt::WA_TransparentForMouseEvents);

    initControl();
}

DrawWidget::~DrawWidget()
{
    delete ui;
}

void DrawWidget::setRect(int x, int y, int width, int height)
{
    _rect.setRect(x, y, width, height);
}

void DrawWidget::setRect2(int x, int y, int width, int height)
{
    _rect2.setRect(x, y, width, height);
    LOG << _rect << _rect2;
}

void DrawWidget::clearListRect()
{
    _listRect.clear();
    update();
}

void DrawWidget::setListRect(QList<QRect> listRect)
{
    _listRect = listRect;
    update();
}

void DrawWidget::initControl()
{

    ::SetWindowPos(HWND(this->winId()), HWND_TOPMOST, 0, 0, 0, 0, SWP_NOMOVE | SWP_NOSIZE | SWP_SHOWWINDOW);
}

void DrawWidget::paintEvent(QPaintEvent *event)
{
    QPainter painter(this);

    drawSelectRect(&painter);
    drawListRect(&painter);
}

void DrawWidget::drawSelectRect(QPainter *painter)
{
    painter->save();

    painter->setPen(QPen(_colorRect, 4));
    painter->drawRect(_rect);
    painter->drawRect(_rect2);

    painter->restore();
}

void DrawWidget::drawListRect(QPainter *painter)
{
    painter->save();

    painter->setPen(QPen(Qt::white, 2));
    for(int index = 0; index < _listRect.size(); index++)
    {
        painter->drawRect(_rect.x() + _listRect.at(index).x(),
                          _rect.y() + _listRect.at(index).y(),
                          _listRect.at(index).width(),
                          _listRect.at(index).height());

    }
    painter->setPen(QPen(Qt::blue, 2));
    for(int index = 0; index < _listRect.size(); index++)
    {
        painter->drawRect(_rect2.x() + _listRect.at(index).x(),
                          _rect2.y() + _listRect.at(index).y(),
                          _listRect.at(index).width(),
                          _listRect.at(index).height());
    }
    painter->restore();
}

Qt代码:FindDifferenceWidget

该类的主要作用:

  • 作为主窗口提供一个套入找茬游戏界面的区域;
  • 提供可以动态选取抓取区域的控件;
  • 提供微调控件,微调应用窗口,并且让DrawWidget窗口与透明区域位置同步改变;
  • 提供识别触发按钮,将识别结果反馈到DrawWidget;
  • 清空按钮,将识别的结果进行清空,也就是删除识别结果的矩形;

Ui界面

项目实战:Qt+OpenCV大家来找茬(Qt抓图,穿透应用,识别左右图区别,框选区别,微调位置)

; FindDifferenceWidget.h

#ifndef FINDDIFFERENCEWIDGET_H
#define FINDDIFFERENCEWIDGET_H

#include
#include
#include
#include
#include
#include "FindDifferenceManager.h"
#include "DrawWidget.h"
#include

namespace Ui {
class FindDifferenceWidget;
}

class FindDifferenceWidget : public QWidget
{
    Q_OBJECT

public:
    explicit FindDifferenceWidget(QWidget *parent = 0);
    ~FindDifferenceWidget();

protected:
    void initControl();
    void updateGameRect();

protected slots:
    void slot_initControl();

protected:
    void paintEvent(QPaintEvent *event);
    void resizeEvent(QResizeEvent *event);
    void moveEvent(QMoveEvent *event);
    void closeEvent(QCloseEvent *event);

protected slots:
    void slot_valueChanged(int value);

protected slots:
    void slot_findResult(bool result, QList<QRect> listRect = QList<QRect>());

private slots:
    void on_pushButton_do_clicked();
    void on_pushButton_up_clicked();
    void on_pushButton_left_clicked();
    void on_pushButton_right_clicked();
    void on_pushButton_down_clicked();
    void on_pushButton_clear_clicked();

private:
    Ui::FindDifferenceWidget *ui;

private:
    FindDifferenceManager *_pFindDifferenceManager;
    QRect _rectGame;
    QRect _rectApplication;
    int _captionHeigh;
    int _margin;

    DrawWidget *_pDrawWidget;
};

#endif

FindDifferenceWidget.cpp

#include "FindDifferenceWidget.h"
#include "ui_FindDifferenceWidget.h"

#include
#include
#include
#include
#include

#include
#include

#define LOG qDebug()<<__FILE__<<__LINE__<<QDateTime::currentDateTime().toString("yyyy-MM-dd hh:mm:ss:zzz")

FindDifferenceWidget::FindDifferenceWidget(QWidget *parent) :
    QWidget(parent),
    ui(new Ui::FindDifferenceWidget),
    _pFindDifferenceManager(0),
    _pDrawWidget(0)
{
    ui->setupUi(this);

    QString version = "v1.0.0";
    setWindowTitle(QString("大家来找茬(仅供学习Qt+OpenCV实战项目) Demo %1(作者:长沙红胖子 QQ:21497936 WX:15173255813 blog:hpzwl.blog.csdn.net").arg(version));

    resize(1230, 785);

    initControl();

    QTimer::singleShot(0, this, SLOT(slot_initControl()));
}

FindDifferenceWidget::~FindDifferenceWidget()
{
    delete ui;
}

void FindDifferenceWidget::initControl()
{

    ::SetWindowPos(HWND(this->winId()), HWND_TOPMOST, 0, 0, 0, 0, SWP_NOMOVE | SWP_NOSIZE | SWP_SHOWWINDOW);

    _pFindDifferenceManager = new FindDifferenceManager();
    connect(_pFindDifferenceManager, SIGNAL(signal_findResult(bool,QList<QRect>)),
            this, SLOT(slot_findResult(bool,QList<QRect>)));

    _captionHeigh = 26;
    _margin = 3;
    updateGameRect();

    _pDrawWidget = new DrawWidget();
    _pDrawWidget->show();

    connect(ui->spinBox_image1X, SIGNAL(valueChanged(int)),
            this, SLOT(slot_valueChanged(int)));
    connect(ui->spinBox_image1Y, SIGNAL(valueChanged(int)),
            this, SLOT(slot_valueChanged(int)));
    connect(ui->spinBox_image1Width, SIGNAL(valueChanged(int)),
            this, SLOT(slot_valueChanged(int)));
    connect(ui->spinBox_image1Height, SIGNAL(valueChanged(int)),
            this, SLOT(slot_valueChanged(int)));

    connect(ui->spinBox_image2X, SIGNAL(valueChanged(int)),
            this, SLOT(slot_valueChanged(int)));
    connect(ui->spinBox_image2Y, SIGNAL(valueChanged(int)),
            this, SLOT(slot_valueChanged(int)));
    connect(ui->spinBox_image2Width, SIGNAL(valueChanged(int)),
            this, SLOT(slot_valueChanged(int)));
    connect(ui->spinBox_image2Height, SIGNAL(valueChanged(int)),
            this, SLOT(slot_valueChanged(int)));

    _pDrawWidget->setRect(ui->spinBox_image1X->value(),
                          ui->spinBox_image1Y->value(),
                          ui->spinBox_image1Width->value(),
                          ui->spinBox_image1Height->value());

    _pDrawWidget->setRect2(ui->spinBox_image2X->value(),
                           ui->spinBox_image2Y->value(),
                           ui->spinBox_image2Width->value(),
                           ui->spinBox_image2Height->value());

}

void FindDifferenceWidget::updateGameRect()
{
    _rectApplication = QRect(-_margin,
                             -_captionHeigh,
                             rect().width() + _margin*2,
                             rect().height() + _captionHeigh + _margin);
    _rectGame = QRect(_margin, rect().height() - _margin - 780, 1032, 780);
}

void FindDifferenceWidget::slot_initControl()
{
    _pDrawWidget->setGeometry(ui->frame_mask->mapToGlobal(QPoint(0, 0)).x(),
                              ui->frame_mask->mapToGlobal(QPoint(0, 0)).y(),
                              ui->frame_mask->width(),
                              ui->frame_mask->height());

}

void FindDifferenceWidget::paintEvent(QPaintEvent *event)
{
#if 1
    QRegion r1(_rectApplication);
    QRegion r2(_rectGame);
    QRegion r3 = r1 - r2;
    setMask(r3);
#endif
    QWidget::paintEvent(event);
}

void FindDifferenceWidget::resizeEvent(QResizeEvent *event)
{

    updateGameRect();

    if(_pDrawWidget)
    {
        _pDrawWidget->setGeometry(ui->frame_mask->mapToGlobal(QPoint(0, 0)).x(),
                                  ui->frame_mask->mapToGlobal(QPoint(0, 0)).y(),
                                  ui->frame_mask->width(),
                                  ui->frame_mask->height());
    }
}

void FindDifferenceWidget::moveEvent(QMoveEvent *event)
{
    if(_pDrawWidget)
    {
        _pDrawWidget->setGeometry(ui->frame_mask->mapToGlobal(QPoint(0, 0)).x(),
                                  ui->frame_mask->mapToGlobal(QPoint(0, 0)).y(),
                                  ui->frame_mask->width(),
                                  ui->frame_mask->height());
    }
    LOG << geometry();
}

void FindDifferenceWidget::closeEvent(QCloseEvent *event)
{
    _pDrawWidget->hide();
    _pDrawWidget->deleteLater();
}

void FindDifferenceWidget::slot_valueChanged(int value)
{
    _pDrawWidget->setRect(ui->spinBox_image1X->value(),
                          ui->spinBox_image1Y->value(),
                          ui->spinBox_image1Width->value(),
                          ui->spinBox_image1Height->value());

    _pDrawWidget->setRect2(ui->spinBox_image2X->value(),
                           ui->spinBox_image2Y->value(),
                           ui->spinBox_image2Width->value(),
                           ui->spinBox_image2Height->value());
    _pDrawWidget->update();

    QSpinBox *pSpinBox = dynamic_cast<QSpinBox *>(sender());
    if(pSpinBox == ui->spinBox_image1Width)
    {
        ui->spinBox_image2Width->setValue(value);
    }else if(pSpinBox == ui->spinBox_image2Width)
    {
        ui->spinBox_image1Width->setValue(value);
    }else if(pSpinBox == ui->spinBox_image1Height)
    {
        ui->spinBox_image2Height->setValue(value);
    }else if(pSpinBox == ui->spinBox_image2Height)
    {
        ui->spinBox_image1Height->setValue(value);
    }
}

void FindDifferenceWidget::slot_findResult(bool result, QList<QRect> listRect)
{
    if(result)
    {
        LOG << listRect;
        _pDrawWidget->setListRect(listRect);
    }
}

void FindDifferenceWidget::on_pushButton_do_clicked()
{
    _pDrawWidget->clearListRect();

    QElapsedTimer elapsedTimer;
    elapsedTimer.start();
    while(elapsedTimer.elapsed() < 500)
    {
        qApp->processEvents();
    }

    QScreen * pScreen = QApplication::primaryScreen();
    QImage gameImage = pScreen->grabWindow(QApplication::desktop()->winId(),
                                           ui->frame_mask->mapToGlobal(QPoint(0, 0)).x(),
                                           ui->frame_mask->mapToGlobal(QPoint(0, 0)).y(),
                                           ui->frame_mask->width(),
                                           ui->frame_mask->height()).toImage();
    QImage image1 = gameImage.copy(ui->spinBox_image1X->value(),
                                   ui->spinBox_image1Y->value(),
                                   ui->spinBox_image1Width->value(),
                                   ui->spinBox_image1Height->value());
    QImage image2 = gameImage.copy(ui->spinBox_image2X->value(),
                                   ui->spinBox_image2Y->value(),
                                   ui->spinBox_image2Width->value(),
                                   ui->spinBox_image2Height->value());
    _pFindDifferenceManager->slot_findDiffrence(image1, image2);
}

void FindDifferenceWidget::on_pushButton_up_clicked()
{
    setGeometry(geometry().x(), geometry().y() - 1, geometry().width(), geometry().height());
}

void FindDifferenceWidget::on_pushButton_left_clicked()
{
    setGeometry(geometry().x() - 1, geometry().y(), geometry().width(), geometry().height());
}

void FindDifferenceWidget::on_pushButton_right_clicked()
{
    setGeometry(geometry().x() + 1, geometry().y(), geometry().width(), geometry().height());
}

void FindDifferenceWidget::on_pushButton_down_clicked()
{
    setGeometry(geometry().x(), geometry().y() + 1, geometry().width(), geometry().height());
}

void FindDifferenceWidget::on_pushButton_clear_clicked()
{
    _pDrawWidget->setListRect(QList<QRect>());
}

OpenCV代码:FindDifferenceManager

识别不同处代码类功能:

  • 引入OpenCV头文件和库文件
  • 提供接口输入2个同样大小的矩形、阈值和能识别的最小矩形。
  • 核心算法:先灰度图->阈值化->寻找边界->识别最小矩形
    (最开始算法进行了 滤波,闭运算,因为实际有可能存在两幅图截屏就有色差导致灰度图色差小,实际有可能原图色差较小导致灰度色差较小,实际有可能不同的点较少导致滤波去掉了这些点,以上三种经过测试都是存在的)。

FindDifferenceManager.h

#ifndef FINDDIFFERENCEMANAGER_H
#define FINDDIFFERENCEMANAGER_H

#include
#include

#include "opencv/highgui.h"
#include "opencv/cxcore.h"
#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/opencv.hpp"
#include "opencv2/xphoto.hpp"
#include "opencv2/dnn/dnn.hpp"

#include
#include
#include
#include
#include
#include

class FindDifferenceManager : public QObject
{
    Q_OBJECT
public:
    explicit FindDifferenceManager(QObject *parent = 0);

private:
    bool getRunning() const;

signals:
    void signal_findResult(bool result, QList<QRect> listRect = QList<QRect>());

public slots:
    void slot_start();
    void slot_stop();

public slots:
    void slot_findDiffrence(QImage image, QImage image2, int thresh = 20, QRect minRect = QRect(0, 0, 4, 4));

protected:
    cv::Mat image2Mat(QImage image);

private:
    bool _running;
};

#endif

FindDifferenceManager.cpp

#include "FindDifferenceManager.h"
#include

#include
#include

#define LOG qDebug()<<__FILE__<<__LINE__<<QDateTime::currentDateTime().toString("yyyy-MM-dd hh:mm:ss:zzz")

FindDifferenceManager::FindDifferenceManager(QObject *parent)
    : QObject(parent),
      _running(false)
{
    qRegisterMetaType<QList<QRect>>("QList ");
}

bool FindDifferenceManager::getRunning() const
{
    return _running;
}

void FindDifferenceManager::slot_start()
{
    if(_running)
    {
        LOG << "Failed to" << __FUNCTION__ << ", it's already running.";
        return;
    }
    _running = true;
}

void FindDifferenceManager::slot_stop()
{
    if(!_running)
    {
        LOG << "Failed to" << __FUNCTION__ << ", it's not running.";
        return;
    }
    _running = false;
}

void FindDifferenceManager::slot_findDiffrence(QImage image, QImage image2, int thresh, QRect minRect)
{
    QList<QRect> listRect;

    cv::Mat srcMat = image2Mat(image);
    cv::Mat srcMat2 = image2Mat(image2);

    if ((srcMat.rows != srcMat2.rows) || (srcMat.cols != srcMat2.cols))
    {
        emit signal_findResult(false);
    }

    cv::Mat srcMatGray;
    cv::Mat srcMat2Gray;

    cv::cvtColor(srcMat, srcMatGray, cv::COLOR_BGR2GRAY);
    cv::cvtColor(srcMat2, srcMat2Gray, cv::COLOR_BGR2GRAY);

    cv::Mat diffMatGray;

    cv::subtract(srcMatGray, srcMat2Gray, diffMatGray, cv::Mat(), CV_16SC1);

    cv::Mat diffAbsMatGray = cv::abs(diffMatGray);

    diffAbsMatGray.convertTo(diffAbsMatGray, CV_8UC1, 1, 0);

#if 0

    for(int row = 0; row < diffAbsMatGray.rows; row++)
    {
        for( int col = 0; col < diffAbsMatGray.cols; col++)
        {
            if(diffAbsMatGray.at<uchar>(row, col) < 3)
            {
                diffAbsMatGray.at<uchar>(row, col) = 0;
            }else{
                diffAbsMatGray.at<uchar>(row, col) = diffAbsMatGray.at<uchar>(row, col) - 5;
            }
        }
    }
#endif

    cv::Mat threshMat;

    cv::threshold(diffAbsMatGray, threshMat, thresh, 255, cv::THRESH_BINARY);

    cv::Mat mdianMat;
#if 0

    cv::medianBlur(threshMat, mdianMat, 3);
    cv::imshow("6", mdianMat);
#else
    mdianMat = threshMat.clone();
#endif

    cv::Mat closeMat;
#if 0

    cv::Mat kernel = cv::getStructuringElement(cv::MORPH_RECT, cv::Size(3, 3), cv::Point(-1, -1));
    cv::morphologyEx(mdianMat, closeMat, cv::MORPH_CLOSE, kernel, cv::Point(-1, -1), 2, cv::BORDER_REPLICATE);
#else
    closeMat = mdianMat.clone();
#endif

    std::vector<std::vector<cv::Point>> contours;
    std::vector<cv::Vec4i> hierarchy;
    cv::findContours(closeMat,
                     contours,
                     hierarchy,
                     CV_RETR_EXTERNAL,
                     CV_CHAIN_APPROX_SIMPLE,
                     cv::Point(0, 0));
    std::vector<std::vector<cv::Point>> contoursPoly(contours.size());

    for(int index = 0; index < contours.size(); index++)
    {
        cv::approxPolyDP(cv::Mat(contours[index]), contoursPoly[index], 5, true);
        cv::Rect rect = cv::boundingRect(cv::Mat(contoursPoly[index]));

#if 0

        if(rect.width < minRect.width() || rect.height < minRect.height())
        {
            continue;
        }
#endif
        listRect.append(QRect(rect.x, rect.y, rect.width, rect.height));

    }

    emit signal_findResult(true, listRect);
}

cv::Mat FindDifferenceManager::image2Mat(QImage image)
{
    cv::Mat mat;
    switch(image.format())
    {
    case QImage::Format_ARGB32:
    case QImage::Format_RGB32:
    case QImage::Format_ARGB32_Premultiplied:
        mat = cv::Mat(image.height(), image.width(), CV_8UC4, (void*)image.constBits(), image.bytesPerLine());
        cv::cvtColor(mat, mat, CV_BGRA2BGR);
        break;
    case QImage::Format_RGB888:
        mat = cv::Mat(image.height(), image.width(), CV_8UC3, (void*)image.constBits(), image.bytesPerLine());
        cv::cvtColor(mat, mat, CV_BGR2RGB);
        break;
    case QImage::Format_Indexed8:
    case QImage::Format_Grayscale8:
        mat = cv::Mat(image.height(), image.width(), CV_8UC1, (void*)image.constBits(), image.bytesPerLine());
        break;
    default:
        qDebug() << __FILE__ << __LINE__ << "error to image2Mat !!!  imge.format =" << image.format();
    }
    return mat;
}

若该文为原创文章,转载请注明出处
本文章博客地址https://hpzwl.blog.csdn.net/article/details/124768637

Original: https://blog.csdn.net/qq21497936/article/details/124768637
Author: 长沙红胖子Qt
Title: 项目实战:Qt+OpenCV大家来找茬(Qt抓图,穿透应用,识别左右图区别,框选区别,微调位置)

原创文章受到原创版权保护。转载请注明出处:https://www.johngo689.com/638197/

转载文章受原作者版权保护。转载请注明原作者出处!

(0)

大家都在看

亲爱的 Coder【最近整理,可免费获取】👉 最新必读书单  | 👏 面试题下载  | 🌎 免费的AI知识星球