ESP32-CAM and OpenCV to read an analog gauge / dial

ESP32-CAM and OpenCV to read an analog gauge / dial
16 January 2021 2 Comments Bez kategorii Bartłomiej Szymański

Introduction

There are many sensors and solutions for measuring physical parameters and values ​​on the market. However, most of these sensors require interference with the system (usually it needs physical contact with the measured body). For example: measuring the pressure of a liquid or gas in a device where only a pressure gauge is installed. Due to technical limitations, where you cannot interferre with transmission medium, a typical analog / digital pressure sensor wouldn’t be useful. Fortunately the manufacturer has installed a pressure gauge. Then you have to think about how you can measure this value. This is where image processing comes in handy. By using a camera and even a simple computer that can handle the OpenCV library, it is easy to solve this problem. It is possible to adapt the device to take measure in a similar way to a human being. Losing a bit on the accuracy of the measurement, we gain the possibility of remote control of the set value. Then we can react quickly when a problem occurs, because the system will inform us about it.


Measurement system and components

The system consists of a SoC (System on Chip) in the form of an integrated camera with a fully-fledged microcontroller and a WiFi module, i.e. ESP32-CAM. In the present case, a computer that uses a python-based program and OpenCV library to take screenshots from the camera’s IP address and analyze them.

Block diagram

Pic. 1. Block diagram of the measuring system.

The ESP32-CAM camera works in the IP camera mode, connects to a given WiFi network and then runs its own web server with image sharing. Python-based program using the OpenCV library captures this image, detects the pressure gauge dial, then detects the needle. It determines the position based on the deviation of the needle angle line from the center of the dial.

Profilometer / gauge

Pic. 2. View of the measured profilometer.

ESP32-CAM

Pic. 3. Presented SoC ESP32-CAM with OV2640 cam.

Connection diagram and pinout

Pic. 4. Connection diagram.

The system should be powered from a stable power source with a voltage of 3.3. Programming takes place with the jumper on and the CH340 chip plugged in. Start programming -> press reset -> wait for the program to be loaded into the memory.


Description of the ESP32-CAM program

In "setup()" function set the UART control output and initialize the camera:
  Serial.begin(115200);
  Serial.println();

  {
    using namespace esp32cam;
    Config cfg;
    cfg.setPins(pins::AiThinker);
    cfg.setResolution(hiRes);
    cfg.setBufferCount(2);
    cfg.setJpeg(80);

    bool ok = Camera.begin(cfg);
    Serial.println(ok ? "CAMERA OK" : "CAMERA FAIL");
  }

Initialize the WiFi module in client mode and wait for connection: 
  WiFi.persistent(false);
  WiFi.mode(WIFI_STA);
  WiFi.begin(WIFI_SSID, WIFI_PASS);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
  }

Define the webserver directory paths and individual functions for specific image transmission, then call the start function (server.begin()). 
  Serial.print("http://");
  Serial.println(WiFi.localIP());
  Serial.println("  /cam.bmp");
  Serial.println("  /cam-lo.jpg");
  Serial.println("  /cam-hi.jpg");
  Serial.println("  /cam.mjpeg");

  server.on("/cam.bmp", handleBmp);
  server.on("/cam-lo.jpg", handleJpgLo);
  server.on("/cam-hi.jpg", handleJpgHi);
  server.on("/cam.jpg", handleJpg);
  server.on("/cam.mjpeg", handleMjpeg);

  server.begin();

In main loop call the client session maintenance function. 
void loop()
{
  server.handleClient();
}


Description of python-based program using the OpenCV library

 
The main loop consists of:  
• parameters to improve the measure quality  
• "take_measure ()" function reading the measurement  
• functions displaying the found target pressure gauge dial and needle in the camera image  
• end of the program condition 
while True:
    #Parameters for real gauge
    #min_angle = 45
    #max_angle = 320
    #min_value = 0
    #max_value = 200
    #units = "PSI"
    
    threshold_img = 120 #175
    threshold_ln = 150
    minLineLength = 40
    maxLineGap = 8

    #Distance from center coefficients
    diff1LowerBound = 0.15 
    diff1UpperBound = 0.25

    #Distance from circle coefficients
    diff2LowerBound = 0.5 
    diff2UpperBound = 1.0
        
    img, img2 = take_measure(threshold_img, threshold_ln, minLineLength, 
    maxLineGap, diff1LowerBound, diff1UpperBound, diff2LowerBound, diff2UpperBound)
    cv2.imshow('test',img)
    cv2.imshow('test2',img2)

    if ord('q')==cv2.waitKey(10):
        exit(0)

Function „take_measure()”:
 • Taking a screenshot from camera:
    imgResp=urllib.request.urlopen(url)
    imgNp=np.array(bytearray(imgResp.read()),dtype=np.uint8)
    img=cv2.imdecode(imgNp,-1)
    img2=cv2.imdecode(imgNp,-1)

• Determining image parameters and searching for a circle in the image  (HoughCircles):
    height, width = img.shape[:2]
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)  
    circles = cv2.HoughCircles(gray, cv2.HOUGH_GRADIENT, 1, 20)

• Drawing a center and circle:
        #Draw center and circle
        cv2.circle(img, (x, y), r, (0, 255, 0), 3, cv2.LINE_AA)  # draw circle
        cv2.circle(img, (x, y), 2, (0, 255, 0), 3, cv2.LINE_AA)  # draw center of circle

• Generating scale and angular division on a given circle, then plotting these parameters on the image:
        separation = 10.0 #in degrees
        interval = int(360 / separation)
        p1 = np.zeros((interval,2))  #set empty arrays
        p2 = np.zeros((interval,2))
        p_text = np.zeros((interval,2))
        for i in range(0,interval):
            for j in range(0,2):
                if (j%2==0):
                    p1[i][j] = x + 0.9 * r * np.cos(separation * i * 3.14 / 180) #point for lines
                else:
                    p1[i][j] = y + 0.9 * r * np.sin(separation * i * 3.14 / 180)
        text_offset_x = 10
        text_offset_y = 5
        for i in range(0, interval):
            for j in range(0, 2):
                if (j % 2 == 0):
                    p2[i][j] = x + r * np.cos(separation * i * 3.14 / 180)
                    p_text[i][j] = x - text_offset_x + 1.2 * r * np.cos((separation) * (i+9) * 3.14 / 180) #point for text labels, i+9 rotates the labels by 90 degrees
                else:
                    p2[i][j] = y + r * np.sin(separation * i * 3.14 / 180)
                    p_text[i][j] = y + text_offset_y + 1.2* r * np.sin((separation) * (i+9) * 3.14 / 180)  # point for text labels, i+9 rotates the labels by 90 degrees

        #Lines and labels
        for i in range(0,interval):
            cv2.line(img, (int(p1[i][0]), int(p1[i][1])), (int(p2[i][0]), int(p2[i][1])),(0, 255, 0), 2)
            cv2.putText(img, '%s' %(int(i*separation)), (int(p_text[i][0]), int(p_text[i][1])), 
            cv2.FONT_HERSHEY_SIMPLEX, 0.3,(255,0,0),1,cv2.LINE_AA)

        cv2.putText(img, "Gauge OK!", (50, 50), cv2.FONT_HERSHEY_SIMPLEX, 0.9,(0,255,0),2,cv2.LINE_AA)

• Grayscale and thresholding of the second image. It makes it easier to find the arrow, then call the function to look for lines in this image: 
        gray3 = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)
        maxValue = 255
        
        # Threshold image to take better measurements
        th, dst2 = cv2.threshold(gray3, threshold_img, maxValue, cv2.THRESH_BINARY_INV);

        in_loop = 0
        lines = cv2.HoughLinesP(image=dst2, rho=3, theta=np.pi / 180, threshold=threshold_ln, minLineLength=minLineLength, maxLineGap=maxLineGap)
        final_line_list = []

• Searching for the coordinates of the lines closest to the center of the drawn circle, then checking that they are within the acceptable range (the values ​​of the coefficients are given in the function main) 
        for i in range(0, len(lines)):
            for x1, y1, x2, y2 in lines[i]:
                diff1 = dist_2_pts(x, y, x1, y1)  # x, y is center of circle
                diff2 = dist_2_pts(x, y, x2, y2)  # x, y is center of circle

                if (diff1 > diff2):
                    temp = diff1
                    diff1 = diff2
                    diff2 = temp
                    
                # Check if line is in range of circle
                if (((diff1<diff1UpperBound*r) and (diff1>diff1LowerBound*r) and (diff2<diff2UpperBound*r)) and (diff2>diff2LowerBound*r)):
                    line_length = dist_2_pts(x1, y1, x2, y2)
                    final_line_list.append([x1, y1, x2, y2])
                    in_loop = 1

        if (in_loop == 1):
            x1 = final_line_list[0][0]
            y1 = final_line_list[0][1]
            x2 = final_line_list[0][2]
            y2 = final_line_list[0][3]
            cv2.line(img2, (x1, y1), (x2, y2), (0, 255, 255), 2)
            dist_pt_0 = dist_2_pts(x, y, x1, y1)
            dist_pt_1 = dist_2_pts(x, y, x2, y2)
            if (dist_pt_0 > dist_pt_1):
                x_angle = x1 - x
                y_angle = y - y1
            else:
                x_angle = x2 - x
                y_angle = y - y2

• Determination and calculation of the inclination angle of the needle in relation to the drawn scale and the center of the dial:
            # Finding angle using the arc tan of y/x
            res = np.arctan(np.divide(float(y_angle), float(x_angle)))

            #Converting to degrees
            res = np.rad2deg(res)
            if x_angle > 0 and y_angle > 0:  #in quadrant I
                final_angle = 270 - res
            if x_angle < 0 and y_angle > 0:  #in quadrant II
                final_angle = 90 - res
            if x_angle < 0 and y_angle < 0:  #in quadrant III
                final_angle = 90 - res
            if x_angle > 0 and y_angle < 0:  #in quadrant IV
                final_angle = 270 - res

            cv2.putText(img2, "Indicator OK!", (50, 50), cv2.FONT_HERSHEY_SIMPLEX, 0.9,(0,255,0),2,cv2.LINE_AA)
            print ("Final Angle: ", final_angle)
•  Averaging function of the found circles - determining the average position of the last found circles from the initial circles:
def avg_circles(circles, b):
    avg_x=0
    avg_y=0
    avg_r=0
    for i in range(b):
        avg_x = avg_x + circles[0][i][0]
        avg_y = avg_y + circles[0][i][1]
        avg_r = avg_r + circles[0][i][2]
    avg_x = int(avg_x/(b))
    avg_y = int(avg_y/(b))
    avg_r = int(avg_r/(b))
    return avg_x, avg_y, avg_r

• Function for determining the distance between the center of the circle and the starting point of the line found, also between the center of the circle and the end point of the line found. 
def dist_2_pts(x1, y1, x2, y2):
    return np.sqrt((x2 - x1)**2 + (y2 - y1)**2)


Effects, possible improvements, development prospects and reflections

System testing

Pic. 5. The effect of reading the value on the profilometer

As you can see on the right, there is a successful attempt to detect the pressure gauge dial, while on the left we have a successful attempt to read the position of the pointer. Console on the right shows the read angle. The results are repeatable, error does not exceed 10 degrees.

In general, the algorithms work quite well, certainly well enough for this application. In the case of moving away, moving in, changing the angle of view up to a certain point the program copes quite well with reading the indication. The only modification that may be needed here is to change the length of detected line, because when you zoom in too much the program can also detect a small arrow visible on the profilometer (I think you can find some simple way to get rid of this effect), while moving the camera away from the profilometer at some point the program does not detect the arrow because it is too short. It is understandable due to the finite number of pixels in the image.

However, there is a problem when lighting is very poor, you need to change threshold values ​​and parameter of the line break time (maxLineGap). I think easy solution is to use several IR LEDs, which will evenly illuminate the gauge dial. This is a bit of a problem when we turn the pressure gauge or the photo, the starting point is not in the same place and you will have to assign different angles to the given indication scale. You can quickly and easily solve it, e.g. by drawing a characteristic line on the pressure gauge, or simply set it rigidly so that it does not rotate and shift.

Any other clock or manometer should be easily detected and read, as long as the image is relatively legible (previously I did tests on photos of other manometers), but in order to get an accurate reading, it is worth playing with the parameters for each of them. It may turn out that the manometer has a wider pointer or another small pointer that will distort the measurement. Therefore, it would be worth expanding the program to automatically eliminate such effects. It would also be a good idea to expand the program to read several manometers in one camera image, so that, for example, you do not need to use three cameras for three pressure gauges placed close to each other. A very big problem appears when we have a non-uniform background, then it is easy to trick the program with another circle appearing in the background, it should be corrected. The camera itself is quite good for simple applications, but better image quality, makes it easier to read correctly with a small error.

It is also worth adding that the needle in the profilometer is very thin, so the program was tested in quite difficult conditions. I think you can still try to play with the thresholding parameters in order to obtain higher quality and frequency of reading.


Summary

By the presented solution is very easy to read the parameters of manometers, where we cannot connect to the tested system with a typical sensor. The solution is relatively cheap, because the ESP32-CAM costs about $10 gross. However, for data analysis an ordinary computer with access to the network is sufficient. Ultimately, I would like the device to work in the field connected to the control panel, where is possible to connect via a VPN tunnel. The server standing already in my area would download such an image and process it properly by collecting the data in the database, which would be used by the other software to visualize the readings (i.e. drawing graphs).

Source files:

https://github.com/bartek-szymanski-szyba/ESP32-projects/tree/master/ESP32-CAM-OpenCV-Gauge-read-via-image-processing

Tags
About The Author
Leave Comment
  1. 1

    Ron

    Hi,

    Do you think I can do the same with this kind of meter: https://m.media-amazon.com/images/I/51+U6ikxZLL.jpg

    Trying to read the fuel level of my generator. Will probably need a night vision capable camera since it is in a closed shed.

    Reply
    1. 1

      admin

      Hello Ron,

      I think the same solution with your meter is possible as long as you are using visible light to improve your readings.

      If I’m not mistaken there is something like glass that can block infrared light from typical camera. In any case it’s worth a try.

      Of course share your results, it’s good to know what types of sensors we can read in this way.

      Reply

Leave a reply

Your email address will not be published. Required fields are marked *