基于OpenCv和swing的图片/视频展示Java实现

基于OpenCv和swing实现图片/视频的展示

图片的展示

swing展示图片，多为操作BufferedImage，这里要关注的核心是将Mat转为BufferedImage。

代码如下：

    public Image toBufferedImage(Mat matrix) {
        int type = BufferedImage.TYPE_BYTE_GRAY;
        if (matrix.channels() > 1) {
            type = BufferedImage.TYPE_3BYTE_BGR;
        }
        int bufferSize = matrix.channels() * matrix.cols() * matrix.rows();
        byte[] buffer = new byte[bufferSize];
        matrix.get(0,0,buffer); // get all the pixels
        BufferedImage image = new BufferedImage(matrix.cols(),matrix.rows(),type);
        final byte[] targetPixels = ((DataBufferByte)image.getRaster().getDataBuffer()).getData();
        System.arraycopy(buffer,0,targetPixels,0,buffer.length);
        return image;
    }

comment 1：OpenCV Mat --> BufferedImage AWT ，创建一个byte array用以保存mat 矩阵的像素信息。数组大小为通道数和图片宽/高之积。其中，Mat.get()将所有的元素导入byte数组。最终，图片的光栅信息通过 getDataBuffer()和getDate()组成接收数组，并通过System.arraycopy方法完成填充。实现最终的类型与数据的转移。

comment 2：图片/视频，最终展示的都为BufferedImage，并在JFrame中展示，这里可将toBufferedImage与swing组件配置以展示的部分抽取为类ImageReader。如下：

import org.opencv.core.Mat;

import javax.swing.*;
import java.awt.*;
import java.awt.image.BufferedImage;
import java.awt.image.DataBufferByte;

/**
 * @Author: nya
 * @Description: 图片展示类
 * @Date: Created in 10:55 2018/9/21
 * @Modify by:
 */
public class ImageViewer {
    private JLabel imageView;

    public void show(Mat image,String windowName) {
        setSystemLookAndFeel();
        JFrame frame = createJFrame(windowName);
        Image loadedImage = toBufferedImage(image);
        imageView.setIcon(new ImageIcon(loadedImage));

        frame.pack();
        frame.setLocationRelativeTo(null);
        frame.setVisible(true);
    }

    private JFrame createJFrame(String windowName) {
        JFrame frame = new JFrame(windowName);
        imageView = new JLabel();
        final JScrollPane imageScrollPane = new JScrollPane(imageView);
        imageScrollPane.setPreferredSize(new Dimension(640,480));
        frame.add(imageScrollPane,BorderLayout.CENTER);
        frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
        return frame;
    }

    private void setSystemLookAndFeel() {
        try {
            UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName());
        } catch (ClassNotFoundException e) {
            e.printStackTrace();
        } catch (InstantiationException e) {
            e.printStackTrace();
        } catch (IllegalAccessException e) {
            e.printStackTrace();
        } catch (UnsupportedLookAndFeelException e) {
            e.printStackTrace();
        }
    }

    public Image toBufferedImage(Mat matrix) {
        int type = BufferedImage.TYPE_BYTE_GRAY;
        if (matrix.channels() > 1) {
            type = BufferedImage.TYPE_3BYTE_BGR;
        }
        int bufferSize = matrix.channels() * matrix.cols() * matrix.rows();
        byte[] buffer = new byte[bufferSize];
        matrix.get(0,0,buffer); // get all the pixels
        BufferedImage image = new BufferedImage(matrix.cols(),matrix.rows(),type);
        final byte[] targetPixels = ((DataBufferByte)image.getRaster().getDataBuffer()).getData();
        System.arraycopy(buffer,0,targetPixels,0,buffer.length);
        return image;
    }
}

image Mat operate code：

        String path = "src/main/java/com/opencv/simpleopencvsample/sample/1.jpg"; 
        Mat openFile = null;
        try {
　　　　　　　// openFile just is path to mat 
            openFile = openFile(path);
            Mat clone = openFile.clone();
            System.out.println(openFile);
            Imgproc.resize(openFile,clone,new Size(640,480));
            System.out.println(clone);
            ImageViewer imageViewer = new ImageViewer();
            imageViewer.show(clone,"Loaded image");

        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            // comment ： never forget to release the matrix
            if (openFile != null ) {
                openFile.release();
            }
        }
        System.out.println(openFile);

视频的展示

有了图片展示为蓝本，视频的操作关键在于VideoCapture类的使用，展示部分不过是捕获视频的每一帧转为Mat，基于swing循环顺序展示即可。获取mat推荐采用read()，此方法为grab()/retrieve()的结合体。

为适应视频处理的JFrame宽高，此处自定义设置，主要采用toBufferedImage方法。

VideoCapture -> Mat -> display code:

import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.videoio.VideoCapture;
import org.opencv.videoio.Videoio;

import javax.swing.*;
import java.awt.*;

/**
 * @Author: nya
 * @Description: 视频捕获相关操作类VideoCapture使用
 * @Date: Created in 13:50 2018/9/21
 * @Modify by:
 */
public class VideoCaptureSample {
    static {
        System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
    }
    private JFrame frame;
    private JLabel imageLabel;

    public static void main(String[] args) {
        VideoCaptureSample sample = new VideoCaptureSample();
        sample.initGUI();
        sample.runMainLoop(args);
    }

    private void initGUI(){
        frame = new JFrame("Camera Input Example");
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setSize(555,970);
        imageLabel = new JLabel();
        frame.add(imageLabel);
        frame.setVisible(true);
    }

    private void runMainLoop(String[] args) {
        ImageViewer viewer = new ImageViewer();
        Mat webcamMatImage = new Mat();
        Image tempImage;
        VideoCapture capture = new VideoCapture("src/main/java/com/opencv/simpleopencvsample/sample/1.mp4");
        capture.set(Videoio.CAP_PROP_FRAME_WIDTH,550);
        capture.set(Videoio.CAP_PROP_FRAME_HEIGHT,960);
        if (capture.isOpened()) {
            while (true) {
                capture.read(webcamMatImage);
                if (!webcamMatImage.empty()) {
                    tempImage = viewer.toBufferedImage(webcamMatImage);
                    ImageIcon imageIcon = new ImageIcon(tempImage,"Captured video");
                    imageLabel.setIcon(imageIcon);
                    frame.pack();
                } else {
                    System.out.println(" --- Frame not captured -- Break !");
                    break;
                }
            }
        } else {
            System.out.println("Couldn't open capture.");
        }
    }
}

常见异常 VideoCapture-isOpened返回false

　　实际测试中，存在视频路径正常，isOpened()一直返回false的问题。

　　这是因为视频处理类VideoCapture位于opencv_videoio模块，使用该类时需在运行时加载预先构建的opencv_ffmpeg * .dll / so。该模块如果加载成功，ffmpeg可用于解码/编码视频；否则，使用其它API。在排除路径问题后，仍然无法读取视频则多为此情况。

　　解决方法很简单，将动态库dll/so导入管理员/root 权限下配置的java.library.path路径下即可。