When conducting web scraping, it can sometimes be more straightforward to identify a value by locating its sibling first. With Python and Beautifulsoup, we can utilize the find() and find_all() methods or CSS selectors along with the select() method to find element siblings efficiently and accurately. This approach is essential for extracting data seamlessly from complex web pages. For those looking to enhance their web scraping capabilities with powerful tools, consider exploring the best web scraping API, which offers a comprehensive suite of features designed to simplify data extraction tasks from a wide array of websites, regardless of their complexity. By integrating such an API into your BeautifulSoup projects, you can significantly improve the accuracy and speed of your web scraping operations, making your data-gathering process more efficient.
import bs4
soup = bs4.BeautifulSoup("""
<span class="price">Price including shipping:</span>
<span>83.13</span>
""")
# using find() method:
soup.find('span', class_="price").find_next_sibling('span').text
"83.13"
# using CSS selectors and `~` notation:
soup.select('.price ~ span')[0].text
# or
soup.select_one('.price ~ span').text
"83.13"