Automating Opera browser with Selenium WebDriver and Python. The right way to automate a web application is, certainly, to understand how this application works, by using burp (see “Burp Suite Free Edition and NTLM authentication in ASP.net applications“) for example, retrieve all necessary requests and learn how to use them.

However, this is sometimes so difficult and confusing, especially when the site owners are actively fighting with automation attempts. In this case, you may want to automate somehow the work on the highest level, with the graphical elements of the site. You can view them as some pictures, as SikuliX does (“SikuliX: the last chance for automation“, or as some elements of code, as Selenium IDE does.

But it would much better to control the browser itself from the Python scripts. And this can be done with Selenium WebDriver.

Guys from Guru99 have wrote me a letter about their free course about Selenium.

This course looks pretty good. In the first part you can find a detailed description of Selenium components and the project history. After reading this, I finally decided that the Selenium WebDriver suited me best. Examples of WebDriver usage are written in Java, and I wanted to use Python. But since the calls are the same, lessons of this course are still more than useful.

And despite the fact that Selenium is usually used with Firefox, I tried to use it with Chromium-based Opera. This browser is very popular in Russia because of some unique features. 😉

Installing Selenium:

$ pip install selenium
Collecting selenium
  Downloading selenium-3.5.0-py2.py3-none-any.whl (921kB)
    100% |████████████████████████████████| 921kB 665kB/s 
Installing collected packages: selenium
Successfully installed selenium-3.5.0

Selecting and downloading  operadriver_linux64.zip from https://github.com/operasoftware/operachromiumdriver/releases

Putting the file in operadriver_linux64/operadriver to my python project folder:

$ ls operadriver_linux64/
operadriver sha-512_sum

Launching my opera-developer with the port number:

$ opera-developer --remote-debugging-port=1212

And here is the code for controlling Opera browser:

from selenium import webdriver
from selenium.webdriver.chrome import service

from bs4 import BeautifulSoup


webdriver_service = service.Service('operadriver_linux64/operadriver')
webdriver_service.start()

capabilities = { 'operaOptions': { 'debuggerAddress': "localhost:1212" }}

driver = webdriver.Remote(webdriver_service.service_url, capabilities)

driver.get('https://www.google.com/')

input_txt = driver.find_element_by_name('q')
input_txt.send_keys('operadriver\n')

soup = BeautifulSoup(driver.page_source, 'html.parser')

print(soup.title.string)
print("---")
for site in soup.find_all('h3'):
    for child in site.children:
        print(child.string)
        print(child['href'])

driver.quit()

The driver took an active tab of Opera browser, opened www.google.com in it, found text field on the page, typed “operadriver” there and made a search request.

Then I’ve got full page code and was able to find the search results using Beautiful Soup (pip install beautifulsoup4).

Output:

operadriver - Google Search
---
GitHub - operasoftware/operachromiumdriver: OperaDriver for ...
https://github.com/operasoftware/operachromiumdriver
Releases · operasoftware/operachromiumdriver · GitHub
https://github.com/operasoftware/operachromiumdriver/releases
OperaDriver should look for the Opera executable on the PATH by ...
https://github.com/operasoftware/operachromiumdriver/issues/9
operadriver - npm
https://www.npmjs.com/package/operadriver
OperaDriver Released | Official Selenium Blog
https://seleniumhq.w_ordpress.com/2011/02/09/operadriver_released/
OperaDriver
https://seleniumhq.github.io/selenium/docs/api/java/org/openqa/selenium/opera/OperaDriver.html
Selenium WebDriver and Opera Driver - Stack Overflow
https://stackoverflow.com/questions/24719270/selenium-webdriver-and-opera-driver
Chocolatey Gallery | Selenium Opera Driver 2.29
https://chocolatey.org/packages/selenium-opera-driver
Dev.Opera — OperaDriver Is Now a Part of Selenium and Has ...
https://dev.opera.com/blog/operadriver-now-a-part-of-selenium-and-experimental-android-support-2/
Maven Repository: com.opera » operadriver » 0.7.3
https://mvnrepository.com/artifact/com.opera/operadriver/0.7.3
Searches related to operadriver

As you can see, it works. You can deal with the WebDriver and automate some specific tasks Starting point for documentation: https://github.com/operasoftware/operachromiumdriver/blob/master/docs/desktop.md

However, some problems were found:

1. Sometimes it become impossible to restart the script, if the browser tab was left opened:

selenium.common.exceptions.WebDriverException: Message: session not created exception
from unknown error: failed to close UI debuggers

You need to close the tab manually, and then everything works.

It would be possible for the script to close the tabs automatically, but you can work with the browser GUI elements only by using keystroke emulation and here is the problem 2.

2. Pressing special keys, including Keys.CONTROL doesn’t work. I tried this ways:

from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
ActionChains(driver).key_down(Keys.CONTROL).send_keys("t").key_up(Keys.CONTROL).perform()
driver.find_element_by_tag_name('body').send_keys(Keys.CONTROL + 't')

But ActionChains mechanism itself works. I can click on Google logo for example:

ActionChains(driver).move_to_element(driver.find_element_by_id('hplogo')).click().perform()

I’ve been googling solution for a long time, but  still can’t get it work.