This article was published as a part of the Data Science Blogathon
How Python can help you beat the online quiz?
Forming a team and entering a table quiz is terrific fun. If the table quiz is also in the Pub, it can make for a great night out. In the office, we run an online quiz every Friday and that adds a bit of fun. Last Friday, I thought could Python help me? Because my scores were and are abysmal. I hoped my Python skills would be better, but I haven’t written a line of code in 6 months. So let’s start analyzing weekly pub quiz!
The weekly office quiz
We have our quizmaster, who sends out the URL on Friday morning and collects the scores from the team as we make our attempt. As we give our scores to the quizmaster, those scores get into the league table, and during our team meeting, we review the leader board. All good fun!
The quiz is from joe.ie
Here is an example:
The quiz consists of 25 questions in 5 rounds ( 5 per round ), each round drawn from a different topic. Such as general knowledge, sport, flags, geography, or cinema, and so forth.
A question has three possible answers (multi-choice). So once you select your preferred solution, the site immediately tells you if you are right or wrong. If you are wrong, it also provides you with the answer. Consider the screenshot below.
Once a contestant has completed all 25 questions, the site gives the result. See the screenshot below. The contestant sends this screenshot to the quizmaster as evidence of participation, requesting inclusion on the leaderboard. Naturally, the quizmaster doesn’t always view that request favourably as there is also a handicap system where points are deducted or added for team-based contributions during the week.
So now you can see how the quiz works, and knowing what we know got me thinking about Python. As I said, my scores are terrible. “I got 7 of 25 right”. So my Python is likely not much better, but we will see.
Getting Python to do the quiz for us
If you like a good Saturday afternoon brain teaser, rather than the office quiz on a Friday, then working with the site, figuring out how it works is a great puzzle and was fun to solve. So after I cracked my Sudoku and played a few hands of Solitaire on the iPad Pro with Magic Keyboard, I was ready. All I needed was a pot of coffee brewing.
I added the Jupyter Notebook to my github repository, so do not worry because now you have all the code. My solution also writes an Excel file; it won’t be the office without Excel files, which is also in the repository as a CSV file.
What do we need?
For me, I powered up my Mac Mini M1, which has the Chrome browser installed. My IDE and Python distribution are always Anaconda for Mac, and it works smoothly with the M1 chip. I use the Jupyter Notebook for this kind of challenge, but you could use VS Code or Spyder as an IDE. These choices are yours, but choose the ones you know and avoid making a double challenge — wrangling an unfamiliar tool and a brain teaser will be frustrating.
Next, you need to code in Python with some magic libraries
import time from bs4 import BeautifulSoup from selenium import webdriver
quiz = "https://www.joe.ie/quiz/joe-friday-pub-quiz-week-254-727578"
driver = webdriver.Chrome('/Applications/chromedriver')
driver.get(quiz);
time.sleep(5)
I used the time module, bs4 for BeautifulSoup, and the webdriver component of the selenium package. You do need to download the ChromeDriver and have that in your application folder. Be careful to match the Chrome version with the associated Driver package or get an error. Using the webdriver Chrome method allowed me to automate a Chrome browser instance and call up the quiz. Now we are cooking!
soup = BeautifulSoup(driver.page_source, 'lxml') xpath = "//div[@data-wpvq-answer="
Having retrieved the quiz from the given URL, we transfer the page source to BeautifulSoup, which represents the page content in a tree structure for us. Then comes the tricky piece we have to examine the page source using the inspector in Chrome. Developer tools please readers!
I made a variable called XPath and sent an initial value of “//div[@data-wpvq-answer=”, and you will notice, from the screenshot above, that it is visible in the page source. However, each ‘data-wpvq-answer’ has a five-digit code that is different for each question and quiz.
Next, we need to get the questions and possible answers, and we use find_all() to create Python lists of each. We have lists of questions, answers, and radio input boxes.
questions = soup.find_all("div", class_="wpvq-question-label")
answers = soup.find_all("label", class_="vq-css-label")
checks = soup.find_all("input", class_="vq-css-checkbox")
The value for class_ is again retrieved from inspecting the page source in Chrome and this is trial and error.
This next bit is a bit tricky
count = 0
ans = []
qs = {}
qq = []
for q in questions:
relanswers = answers[count*3: (count*3)+3]
atts = checks[count*3].attrs
path = xpath+atts['data-wpvq-answer']+"]"
#print(atts)
driver.find_element_by_xpath(path).click()
time.sleep(1)
for cho in relanswers:
ans.append(cho.text.strip('n'))
count = count +1
qs['Question'] = q.text
qs['Answers'] = ans
ans=[]
qq.append(qs)
qs = {}
qs is a Python dictionary that gets each question (25) and the three possible answers. Those values are already available from the Python lists we generated in the previous step. To get a list of dictionaries each qs is added to the list qq in turn.
To get the answer, I use the driver.find_element_by_xpath with the click method. We know that clicking any question provides the solution. Either the guess is correct, which shows up in Green, or the answer shows in Red. Those seasoned campaigners will know that colours and styling apply to HTML via CSS classes, which is an exciting hint on finding the solutions.
To satisfy those spreadsheet lovers, naturally, I herewith introduce Pandas and Pandas is truly magic!
import pandas as pd
soup = BeautifulSoup(driver.page_source, 'lxml')
rechecks = soup.find_all("div", class_="wpvq-answer")
Now because I went ahead and used Selenium to interact with the page, I ended up with a stale copy of the page source in memory. You’ve got to think these things through or fail. So I loaded the page source into my ‘soup’ object again. Having guessed each question to provoke the answer, I again examined the page source and found the class that helped me see the answers. It’s one thing for the human eye to observe the response, but we need to decode that into information that Python understands.
My variable rechecks, a python list, has all the possible answers with the Green, None, or Red CSS classes attached. If you got to this stage, you probably guessed by now, but I won this one! I was smelling those roses at this stage.
I am afraid the code became a bit more complicated after that
corrected = []
counter = 0
corr = []
for i in range(0,len(questions)):
for g in range(i*3,(i*3)+3):
res = rechecks[g]
dic = res.attrs
#print (dic['class'])
corr.append(dic['class'])
corrected.append(corr)
corr = []
The variable corrected is a Python list containing the site JavaScript-based response to my random clicks, exposing the proper answer. As each question has three possible solutions, we have a list of 25, each entry containing a list of the three responses. If that is hard, and I did this, you have to play an easier Sudoku until you get more practice.
I had to write a function to figure out the correct answer, but I am not sure that I am sorry because I think my Python is getting better as we go!
def checker(x):
possible = x['Answers']
resulst = x['results']
response = ''
for i in range(0,len(possible)):
if "wpvq-answer-true" in resulst[i]:
response = possible[i]
return response
The function takes an entire row of a DataFrame and does some figuring out. A question is a row in the matrix with a column for possible answers, plus a column for the response given on the Quiz to each.
The correct answer has the CSS class ‘wpvq-answer-true’, so we need only find the response (1 of 3) index containing the class and use that index to hook out the correct answer from the list (1 of 3 ) of possible solutions.
df = pd.DataFrame(qq)
df['results'] = corrected
df['answer'] = df.apply(lambda x: checker(x), axis=1)
df.to_csv('joe.csv')
Next, I created a DataFrame (df) using my qq Python list of dictionaries discussed previously. Finally, two computed fields get added — one with the results from the wild guessing and the correct answer based on our detective work. Last, and naturally most important, for the office workers, we create the Excel file. Magic!
driver.quit()
Close and tear down the connection with the Chrome session.
and now……..
Are we right?
Okay, that seemed like a lot of fun, and it was for the most part, except for the Chrome errors, Python errors, and some issue with Chrome adding ‘tttttttttttttttttttttt’ to every line of code and making an absolute mess of things!!!!!! We had bugs as well!
So how do we know if we are right? Well, let’s do some testing. Since I made a convenient list of the questions and the answers, it was relatively straightforward to visit some quizzes and just use the solutions generated by Python. I used three previous quizzes as a sort of backtesting, and well, and yes, you guessed it, I got 25 out of 25 each time.
What Next
So last Friday, I thought could Python help me? Because my scores were and are abysmal. I had hoped my Python skills would be better; I haven’t written a line of code in 6 months, I said. I can’t wait for another 6 months to see how bad things really get!
Well, I enjoyed writing the code and telling my story here. Doing these challenges is more fun than Sudoku, and more challenging than Solitaire, but I guess I’ll be disqualified on Friday when this news breaks. Onwards and upwards to the next challenge.
Cheating seriously reduces everyone’s fun so let’s avoid that!
Image Source
- Image 1: https://unsplash.com?utm_source=medium&utm_medium=referral
- Image 2: https://www.joe.ie/?ref=logo
- Image 3: https://www.joe.ie/quiz/the-joe-friday-pub-quiz-week-262-731875
- Image 4: https://unsplash.com?utm_source=medium&utm_medium=referral
- Image 5: https://unsplash.com/@elisa_ventur?utm_source=medium&utm_medium=referral
- Image 6: https://unsplash.com/@rresenden?utm_source=medium&utm_medium=referral
- Image 7: https://unsplash.com/@rresenden?utm_source=medium&utm_medium=referral
- Image 8: https://unsplash.com/@rresenden?utm_source=medium&utm_medium=referral
