Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
365 views
in Technique[技术] by (71.8m points)

python - how do i get coordinates of image shown in opencv

Sorry but title doesnt really make sense

i am trying to make an ai that clicks on the ball to make it bounce. for context heres a picture of the application enter image description here

in the game when you click the ball it goes up and then comes back down and the aim of the game is to keep it up.

i have writen some code that turns the image into a mask with opencv, heres a picture of the result

enter image description here

what i now need to do is find the location of the ball in pixels/coordinates so i can make the mouse move to it and click it. By the way the ball has a margin on the left and right of it so it doesn't just go strait up and down but left and right too. Also the ball isnt animated,just a moving image.

How would i get the ball location in pixels/coordinates so i can move the mouse to it.

heres a copy of my code:

import numpy as np
from PIL import ImageGrab
import cv2
import time
import pyautogui


def draw_lines(img,lines):
    for line in lines:
        coords = line[0]
        cv2.line(img, (coords[0], coords[1]), (coords[2], coords[3]), [255,255,255], 3)

def process_img(original_image):
    processed_img = cv2.cvtColor(original_image, cv2.COLOR_BGR2GRAY)
    processed_img = cv2.Canny(processed_img, threshold1=200, threshold2=300)
    vertices = np.array([[0,0],[0,800],[850,800],[850,0]
                         ], np.int32)
    processed_img = roi(processed_img, [vertices])

    # more info: http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_imgproc/py_houghlines/py_houghlines.html
    #                          edges       rho   theta   thresh         # min length, max gap:        
    lines = cv2.HoughLinesP(processed_img, 1, np.pi/180, 180,      20,         15)
    draw_lines(processed_img,lines)
    return processed_img

def roi(img, vertices):
    #blank mask:
    mask = np.zeros_like(img)
    # fill the mask
    cv2.fillPoly(mask, vertices, 255)
    # now only show the area that is the mask
    masked = cv2.bitwise_and(img, mask)
    return masked
def main():
    last_time = time.time()
    while(True):
        screen =  np.array(ImageGrab.grab(bbox=(0,40, 800, 850)))
        new_screen = process_img(screen)
        print('Loop took {} seconds'.format(time.time()-last_time))
        last_time = time.time()
        cv2.imshow('window', new_screen)
        #cv2.imshow('window2', cv2.cvtColor(screen, cv2.COLOR_BGR2RGB))
        if cv2.waitKey(25) & 0xFF == ord('q'):
            cv2.destroyAllWindows()
            break

def mouse_movement():
    ##Set to move relative to where ball is
    pyautogui.moveTo(300,400)
    pyautogui.click();
main()

Sorry if this is confusing but brain.exe has stopped working :( Thanks

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

You can do it like this:

1. crop an image of the ball from a screenshot or so, sth. like

img = cv2.imread("screenshot.jpg")
crop_img = img[y:y+h, x:x+w] # you will have to look for the parameters by trial and error

2. use template matching to look where the ball is in your image

3. get the point in the middle of the resulting rectangle and move your mouse there

I hope this helps, if you need more help on how to achieve any of this feel free to ask


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...