**DIGITAL MEDIA PLANNER APPRENTICE**\n\nBecome a digital media planner apprentice

Bright

bright

HODL 2025, set for 14-15 May 2025 at the Madinat Jumeirah in Dubai, follows the Dubai FinTech Summit and marks a new chapter for Trescon’s flagship event. Formerly known as the World Blockchain Summit, the event has been at the forefront of blockchain innovation for nearly a decade. As the world’s longest-running crypto and Web3 summit series, HODL 2025 will bring together over 5,000 industry leaders, innovators, and investors to explore the technologies and trends that will define the next decade. Join the discussion on the groundbreaking shifts reshaping Web3, crypto, and DeFi. Institutional adoption is accelerating as TradFi ramps up its crypto play, while evolving regulations set the stage for the next compliance era. DeFi 2.0 is breaking boundaries with real-world integrations, and AI-powered blockchain innovations are pushing scalability to new heights alongside next-gen cross-chain interoperability. From RWA tokenization unlocking trillions in liquidity to Web3 gaming emerging as the next digital frontier, and modular blockchain architectures revolutionizing infrastructure—this is where the future of the industry unfolds. Expect hot takes on market trends, the rise of Layer-2 solutions, and blockchain’s expanding role in enterprise innovation HODL 2025 is where the industry’s brightest minds come together to collaborate and explore the real-world impact of blockchain and Web3 technologies. What to Expect at HODL 2025: What to Expect at HODL 2025: Investor & Collaboration Access: Connect with over 500 investors looking to fund blockchain projects and seize opportunities for collaboration with industry leaders and innovators. HODL 2025 is the place to secure funding and form strategic partnerships.


Brand Exposure & Strategic Insights: With 60+ media and listing partners, your brand will gain global visibility. At the same time, discover how emerging blockchain technologies are transforming industries and learn how to leverage them for growth.


Powerhouse Speakers & Networking: Hear from leading experts in blockchain, AI, DeFi, and Web3. Plus, meet entrepreneurs, regulators, and investors from around the world, expanding your network and gaining valuable insights.


Collaborative Discussions: Engage in focused discussions on governance, tokenization, and interoperability, gaining practical insights to solve real-world challenges in the blockchain space. Investor & Collaboration Access: Connect with over 500 investors looking to fund blockchain projects and seize opportunities for collaboration with industry leaders and innovators. HODL 2025 is the place to secure funding and form strategic partnerships. Investor & Collaboration Access: Connect with over 500 investors looking to fund blockchain projects and seize opportunities for collaboration with industry leaders and innovators. HODL 2025 is the place to secure funding and form strategic partnerships. Investor & Collaboration Access: Brand Exposure & Strategic Insights: With 60+ media and listing partners, your brand will gain global visibility. At the same time, discover how emerging blockchain technologies are transforming industries and learn how to leverage them for growth. Brand Exposure & Strategic Insights: With 60+ media and listing partners, your brand will gain global visibility. At the same time, discover how emerging blockchain technologies are transforming industries and learn how to leverage them for growth. Brand Exposure & Strategic Insights: Powerhouse Speakers & Networking: Hear from leading experts in blockchain, AI, DeFi, and Web3. Plus, meet entrepreneurs, regulators, and investors from around the world, expanding your network and gaining valuable insights. Powerhouse Speakers & Networking: Hear from leading experts in blockchain, AI, DeFi, and Web3. Plus, meet entrepreneurs, regulators, and investors from around the world, expanding your network and gaining valuable insights. Powerhouse Speakers & Networking: Collaborative Discussions: Engage in focused discussions on governance, tokenization, and interoperability, gaining practical insights to solve real-world challenges in the blockchain space. Collaborative Discussions: Engage in focused discussions on governance, tokenization, and interoperability, gaining practical insights to solve real-world challenges in the blockchain space. Collaborative Discussions: There was a time when brands set up individual meetings to pitch their ideas. But at HODL 2025, you’ll be under one roof with not just your target prospects, but thousands of potential collaborators. This is where you can pitch your ideas, collaborate in real-time, and expand your network to include key players from across the blockchain ecosystem. The lead generation process has evolved. Events like HODL bring together both the solution and the user in one space. This creates an ideal environment for forging meaningful connections, driving innovation, and building strategic partnerships that will shape the future of blockchain. As we look forward to the discussions at HODL 2025, Nils Andersen-Röed, Global Head of FIU at Binance, said, “Excited to share insights on blockchain and connect with leaders driving GameFi, NFTs, and Web3 advancements at HODL 2025.” Billal Yamak, Head of Growth GCC, The Sandbox, added, “HODL 2025 is where the future of Web3 and crypto innovation takes shape. I am thrilled to collaborate with pioneers shaping this exciting digital revolution” Alongside Nils Andersen-Röed, the summit will feature a distinguished lineup of speakers, including: Corbin Fraser, CEO, Bitcoin.Com


Robert Crossley, Global Head of Industry and Digital Advisory Services, Franklin Templeton


Gracy Chen, CEO, Bitget


Viktor Fischer, Managing Partner, RockawayX


Talal Tabbaa, CEO, CoinMena


Luther Maday, Global Head of Payments, Algorand Foundation


Dyma Budorin, Co-Founder & CEO, Hacken Corbin Fraser, CEO, Bitcoin.Com Corbin Fraser, CEO, Bitcoin.Com Corbin Fraser Robert Crossley, Global Head of Industry and Digital Advisory Services, Franklin Templeton Robert Crossley, Global Head of Industry and Digital Advisory Services, Franklin Templeton Robert Crossley Gracy Chen, CEO, Bitget Gracy Chen, CEO, Bitget Gracy Chen Viktor Fischer, Managing Partner, RockawayX Viktor Fischer, Managing Partner, RockawayX Viktor Fischer Talal Tabbaa, CEO, CoinMena Talal Tabbaa, CEO, CoinMena Talal Tabbaa Luther Maday, Global Head of Payments, Algorand Foundation Luther Maday, Global Head of Payments, Algorand Foundation Luther Maday Dyma Budorin, Co-Founder & CEO, Hacken Dyma Budorin, Co-Founder & CEO, Hacken Dyma Budorin With its deep focus on blockchain adoption, AI integration, DeFi, NFTs, and institutional investments in Web3, HODL continues to be the premier networking destination for industry stakeholders. For more information, visit: https://bit.ly/4cC9HyM Unlock your FREE access to HODL 2025 (limited time only). Click here to claim! Click here to claim! Click here to claim! This 30th edition of HODL Summit is supported by Platinum Sponsor - Liquid Loans
Gold Sponsors - Tata Consultancy Services | Gofaizen & Sherle
Silver Sponsor - Facephi
Bronze Sponsors - Skygate Network | FMCPAY | Solutions
Exhibitors - SWAG42 | Thesaurum
Startup Exhibitors - Scintilla | Boston Trading Co
Association Partners - Global Blockchain Business Council (GBBC) | D2A2 | DGroup Capital
Experiential Partner - Lazulite Technology Services
Official Media Partner - CNN Business Arabic
Official Digital PR Distribution Partner - ZEX PR WIRE Platinum Sponsor - Liquid Loans Platinum Sponsor Liquid Loans Gold Sponsors - Tata Consultancy Services | Gofaizen & Sherle Gold Sponsors Tata Consultancy Services Gofaizen & Sherle Silver Sponsor - Facephi Silver Sponsor Bronze Sponsors - Skygate Network | FMCPAY | Solutions Bronze Sponsors Exhibitors - SWAG42 | Thesaurum Exhibitors Startup Exhibitors - Scintilla | Boston Trading Co Startup Exhibitors Association Partners - Global Blockchain Business Council (GBBC) | D2A2 | DGroup Capital Association Partners Experiential Partner - Lazulite Technology Services Experiential Partner Official Media Partner - CNN Business Arabic Official Media Partner - Official Digital PR Distribution Partner - ZEX PR WIRE Official Digital PR Distribution Partner For further inquiries, contact: For further inquiries, contact: Shadi DawiSr. Director – PR, Comms., & MediaM: +971 55 498 4989 | E: shadi@tresconglobal.com About HODL About HODL HODL, born from the legacy of the iconic **World Blockchain Summit (**WBS) and organised by Trescon, is the World’s Longest-Running Crypto and Web3 Summit Series with 29 global editions across cities like Dubai, Singapore, and Bangkok. Since 2017, HODL has served as a platform for blockchain deal-making, innovation, and connecting disruptive projects with investors, enterprises, and governments. Join HODL Dubai on 14-15 May 2025, to engage with industry leaders, explore cutting-edge blockchain developments, and secure your place at the forefront of Web3 innovation. HODL Dubai 14-15 May 2025 Visit https://bit.ly/4cC9HyM for more details and ticket information. https://bit.ly/4cC9HyM https://bit.ly/4cC9HyM

HODL 2025: Blockchain’s Brightest Minds. All in Dubai

HODL 2025: les ments més brillants de Blockchain. tot a Dubai

HODL 2025: las mentes más brillantes de Blockchain. todo en Dubai

HODL 2025: Blockchain- ի ամենագեղեցիկ հոգիններ: Բոլորը Dubai- ում

HODL 2025:ブロックチェーンの最も明るい頭脳。

HODL 2025: គំនិតអស្ចារ្យបំផុតនៃ Blockchain ។ ទាំងអស់នៅក្នុងទីក្រុង Dubai

HODL 2025: Blockchain spilgtākie prāti. visi Dubaijā

HODL 2025: Minda paling cerah Blockchain. semua di Dubai

HODL 2025: Blockchain's Brightest Minds. alles in Dubai

HODL 2025: Mendjet më të ndritshme të Blockchain. të gjitha në Dubai

HODL 2025: Найяскравіші уми блокчейна. все в Дубаї

Howdy Hackers, The wait is over—we’re thrilled to announce the finalists and winners of the AI Writing Contest, presented by Bright Data and HackerNoon. This contest, which kicked off on September 2, 2024, put the spotlight on a key player in the world of Artificial Intelligence—data. Writers explored how AI training data is collected, how it shapes performance, and the best tools for sourcing high-quality datasets. With AI being one of the hottest topics in tech, it was no surprise that nearly 300 stories were published under the #ai tag for the duration of the contest, racking up 290K+ page views and over 450 hours of reading time. After a two-week extension, the contest officially wrapped up as 2024 came to a close. Our editors carefully reviewed all submissions and narrowed them down to 10 finalists—all in the running for a share of the $2,500 prize pool as follows: And now, it’s time for the big reveal… 🏆 New to HackerNoon writing contests?
Find out about active contests, participation guidelines, and more at contests.hackernoon.com Let’s meet our finalists! AI Writing Contest: Finalists How to Utilize Bright Data for Sentiment Analysis by @adnanaleeza Beyond Text Embeddings: Addressing the Gaps in RAG Applications for Structured Data Queries by @neo4j How I Scraped YouTube Comments with Bright Data to Understand Customer Sentiment by @ayinketh Can We Train AI to Become More Human-Like? by @ssukhpinder How Bright Data Simplifies Web Scraping/Data Collection for AI Training by @diamondolmd Data Centric AI? Yes, but… by @docligot Let's Build a Free Web Scraping Tool That Combines Proxies and AI for Data Analysis by @aviatorscode2 Is AI Progress Slowing? The Scaling Debate OpenAI Doesn’t Want to Have by @dosseyrichards Weather Image Recognition with AI: Dataset Labeling, Model Training, and Real-world Usecases by @induction The One-Shot Generalization Paradox: Why Generative AI Struggles With New Information by @pawarashishanil From making AI more human-like to scraping YouTube comments for real insights, these finalists are tackling some of the most exciting challenges in AI today. Whether you're interested in data collection techniques, sentiment analysis, or the future of AI scaling, there's a story here for you. Jump in, explore their work, and follow their journey on HackerNoon! After collating votes from our editorial team, we're thrilled to announce the winners of the AI Writing Contest! The Winners of the AI Writing Contest Are… Best Bright Data Story https://hackernoon.com/how-i-scraped-youtube-comments-with-bright-data-to-understand-customer-sentiment?embedable=true TLDR: Overcome the challenges with traditional web scraping process, with the use of Bright Data as an all-in-one efficient tool designed to tackle CAPTCHAs, Honeypot, Rate limiting, Block request, and IP blocks faced during data retrieval. Along with Python libraries like Playwright, to scrap data from YouTube. Congratulations @ayinketh, you’ve won $1000. Best AI Story https://hackernoon.com/the-one-shot-generalization-paradox-why-generative-ai-struggles-with-new-information?embedable=true Generative AI, like GPT-4, is extraordinary at generating text based on vast amounts of data but fails when confronted with new, unfamiliar information. This “One-Shot Generalization Paradox” demonstrates that despite their power, current AI models rely on pre-existing patterns and struggle with novel tasks. We explore the reasons behind this (from transformer architecture limitations to dense vector representations) and look at promising solutions like meta-learning and neuro-symbolic architectures to enable true generalization in AI. Congratulations @pawarashishanil, you’ve won $1000 Best Web Scraping Story https://hackernoon.com/how-bright-data-ai-made-web-data-scrapingcollection-effortless-the-challenges-before-bright-data-ai-solutions?embedable=true As a business owner, student, drop shipper, or corporate worker, you must have faced the risk of going out of business if you lack access to some (if not the most) crucial piece of data. But fear not; there is a solution that helps you unlock the full potential of web data scraping/collection, which I will show you in this article. Congratulations @diamondolmd, you’ve won $500 Congratulations once again to all our winners. Thank you for your hard work! How to Claim Your HackerNoon Writing Contest Prize Contact yes-reply@hackernoon.com and sidra@hackernoon.com and share the email ID/and or wallet address attached to the winning hackernoon account.
We will validate your claim and share a form requesting your details for reward distribution.
You will receive your winnings in 2-4 weeks after completing the form. Please note that you must contact us within 60 days after the winners’ announcement date. Ready to be a winner? Discover active writing contests on HackerNoon and learn how you can earn a spot on our next winners' list at: contests.hackernoon.com Howdy Hackers, The wait is over—we’re thrilled to announce the finalists and winners of the AI Writing Contest , presented by Bright Data and HackerNoon. AI Writing Contest Bright Data This contest, which kicked off on September 2, 2024, put the spotlight on a key player in the world of Artificial Intelligence— data . Writers explored how AI training data is collected, how it shapes performance, and the best tools for sourcing high-quality datasets. data With AI being one of the hottest topics in tech, it was no surprise that nearly 300 stories were published under the #ai tag for the duration of the contest, racking up 290K+ page views and over 450 hours of reading time . After a two-week extension, the contest officially wrapped up as 2024 came to a close. 290K+ page views 450 hours of reading time Our editors carefully reviewed all submissions and narrowed them down to 10 finalists —all in the running for a share of the $2,500 prize pool as follows: finalists $2,500 prize pool And now, it’s time for the big reveal… 🏆 And now, it’s time for the big reveal… 🏆 New to HackerNoon writing contests? Find out about active contests, participation guidelines, and more at contests.hackernoon.com New to HackerNoon writing contests? New to HackerNoon writing contests? New to HackerNoon writing contests? Find out about active contests, participation guidelines, and more at contests.hackernoon.com Find out about active contests, participation guidelines, and more at contests.hackernoon.com contests.hackernoon.com contests.hackernoon.com contests.hackernoon.com Let’s meet our finalists! AI Writing Contest: Finalists How to Utilize Bright Data for Sentiment Analysis by @adnanaleeza How to Utilize Bright Data for Sentiment Analysis by @adnanaleeza How to Utilize Bright Data for Sentiment Analysis @adnanaleeza Beyond Text Embeddings: Addressing the Gaps in RAG Applications for Structured Data Queries by @neo4j Beyond Text Embeddings: Addressing the Gaps in RAG Applications for Structured Data Queries by @neo4j Beyond Text Embeddings: Addressing the Gaps in RAG Applications for Structured Data Queries @neo4j How I Scraped YouTube Comments with Bright Data to Understand Customer Sentiment by @ayinketh How I Scraped YouTube Comments with Bright Data to Understand Customer Sentiment by @ayinketh How I Scraped YouTube Comments with Bright Data to Understand Customer Sentiment @ayinketh Can We Train AI to Become More Human-Like? by @ssukhpinder Can We Train AI to Become More Human-Like? by @ssukhpinder Can We Train AI to Become More Human-Like? @ssukhpinder How Bright Data Simplifies Web Scraping/Data Collection for AI Training by @diamondolmd How Bright Data Simplifies Web Scraping/Data Collection for AI Training by @diamondolmd How Bright Data Simplifies Web Scraping/Data Collection for AI Training @diamondolmd Data Centric AI? Yes, but… by @docligot Data Centric AI? Yes, but… by @docligot Data Centric AI? Yes, but… @docligot Let's Build a Free Web Scraping Tool That Combines Proxies and AI for Data Analysis by @aviatorscode2 Let's Build a Free Web Scraping Tool That Combines Proxies and AI for Data Analysis by @aviatorscode2 Let's Build a Free Web Scraping Tool That Combines Proxies and AI for Data Analysis @aviatorscode2 Is AI Progress Slowing? The Scaling Debate OpenAI Doesn’t Want to Have by @dosseyrichards Is AI Progress Slowing? The Scaling Debate OpenAI Doesn’t Want to Have by @dosseyrichards Is AI Progress Slowing? The Scaling Debate OpenAI Doesn’t Want to Have @dosseyrichards Weather Image Recognition with AI: Dataset Labeling, Model Training, and Real-world Usecases by @induction Weather Image Recognition with AI: Dataset Labeling, Model Training, and Real-world Usecases by @induction Weather Image Recognition with AI: Dataset Labeling, Model Training, and Real-world Usecases @induction The One-Shot Generalization Paradox: Why Generative AI Struggles With New Information by @pawarashishanil The One-Shot Generalization Paradox: Why Generative AI Struggles With New Information by @pawarashishanil The One-Shot Generalization Paradox: Why Generative AI Struggles With New Information @pawarashishanil From making AI more human-like to scraping YouTube comments for real insights, these finalists are tackling some of the most exciting challenges in AI today. Whether you're interested in data collection techniques, sentiment analysis, or the future of AI scaling, there's a story here for you. Jump in, explore their work, and follow their journey on HackerNoon! From making AI more human-like to scraping YouTube comments for real insights, these finalists are tackling some of the most exciting challenges in AI today. Whether you're interested in data collection techniques, sentiment analysis, or the future of AI scaling, there's a story here for you. Jump in, explore their work, and follow their journey on HackerNoon! After collating votes from our editorial team, we're thrilled to announce the winners of the AI Writing Contest! After collating votes from our editorial team, we're thrilled to announce the winners of the AI Writing Contest! After collating votes from our editorial team, we're thrilled to announce the winners of the AI Writing Contest! The Winners of the AI Writing Contest Are… Best Bright Data Story https://hackernoon.com/how-i-scraped-youtube-comments-with-bright-data-to-understand-customer-sentiment?embedable=true https://hackernoon.com/how-i-scraped-youtube-comments-with-bright-data-to-understand-customer-sentiment?embedable=true TLDR: Overcome the challenges with traditional web scraping process, with the use of Bright Data as an all-in-one efficient tool designed to tackle CAPTCHAs, Honeypot, Rate limiting, Block request, and IP blocks faced during data retrieval. Along with Python libraries like Playwright, to scrap data from YouTube. TLDR: Overcome the challenges with traditional web scraping process, with the use of Bright Data as an all-in-one efficient tool designed to tackle CAPTCHAs, Honeypot, Rate limiting, Block request, and IP blocks faced during data retrieval. Along with Python libraries like Playwright, to scrap data from YouTube. Congratulations @ayinketh, you’ve won $1000. Congratulations @ayinketh, you’ve won $1000. Congratulations @ayinketh , you’ve won $1000. @ayinketh Best AI Story https://hackernoon.com/the-one-shot-generalization-paradox-why-generative-ai-struggles-with-new-information?embedable=true https://hackernoon.com/the-one-shot-generalization-paradox-why-generative-ai-struggles-with-new-information?embedable=true Generative AI, like GPT-4, is extraordinary at generating text based on vast amounts of data but fails when confronted with new, unfamiliar information. This “One-Shot Generalization Paradox” demonstrates that despite their power, current AI models rely on pre-existing patterns and struggle with novel tasks. We explore the reasons behind this (from transformer architecture limitations to dense vector representations) and look at promising solutions like meta-learning and neuro-symbolic architectures to enable true generalization in AI. Generative AI, like GPT-4, is extraordinary at generating text based on vast amounts of data but fails when confronted with new, unfamiliar information. This “One-Shot Generalization Paradox” demonstrates that despite their power, current AI models rely on pre-existing patterns and struggle with novel tasks. We explore the reasons behind this (from transformer architecture limitations to dense vector representations) and look at promising solutions like meta-learning and neuro-symbolic architectures to enable true generalization in AI. Congratulations @pawarashishanil, you’ve won $1000 Congratulations @pawarashishanil, you’ve won $1000 Congratulations @pawarashishanil , you’ve won $1000 @pawarashishanil Best Web Scraping Story https://hackernoon.com/how-bright-data-ai-made-web-data-scrapingcollection-effortless-the-challenges-before-bright-data-ai-solutions?embedable=true https://hackernoon.com/how-bright-data-ai-made-web-data-scrapingcollection-effortless-the-challenges-before-bright-data-ai-solutions?embedable=true As a business owner, student, drop shipper, or corporate worker, you must have faced the risk of going out of business if you lack access to some (if not the most) crucial piece of data. But fear not; there is a solution that helps you unlock the full potential of web data scraping/collection, which I will show you in this article. As a business owner, student, drop shipper, or corporate worker, you must have faced the risk of going out of business if you lack access to some (if not the most) crucial piece of data. But fear not; there is a solution that helps you unlock the full potential of web data scraping/collection, which I will show you in this article. Congratulations @diamondolmd, you’ve won $500 Congratulations @diamondolmd, you’ve won $500 Congratulations @diamondolmd , you’ve won $500 @diamondolmd Congratulations once again to all our winners. Thank you for your hard work! Congratulations once again to all our winners. Thank you for your hard work! How to Claim Your HackerNoon Writing Contest Prize Contact yes-reply@hackernoon.com and sidra@hackernoon.com and share the email ID/and or wallet address attached to the winning hackernoon account. We will validate your claim and share a form requesting your details for reward distribution. You will receive your winnings in 2-4 weeks after completing the form. Contact yes-reply@hackernoon.com and sidra@hackernoon.com and share the email ID/and or wallet address attached to the winning hackernoon account. yes-reply@hackernoon.com sidra@hackernoon.com We will validate your claim and share a form requesting your details for reward distribution. You will receive your winnings in 2-4 weeks after completing the form. Please note that you must contact us within 60 days after the winners’ announcement date. Please note that you must contact us within 60 days after the winners’ announcement date. Ready to be a winner? Ready to be a winner? Discover active writing contests on HackerNoon and learn how you can earn a spot on our next winners' list at: contests.hackernoon.com Discover active writing contests on HackerNoon and learn how you can earn a spot on our next winners' list at: contests.hackernoon.com Discover active writing contests on HackerNoon and learn how you can earn a spot on our next winners' list at: contests.hackernoon.com contests.hackernoon.com contests.hackernoon.com

The AI Writing Contest by Bright Data & HackerNoon: Results Announcement 🎉

How I Scraped YouTube Comments with Bright Data to Understand Customer Sentiment

The Role of Data in Machine Learning In today's AI-driven landscape, machine learning, AI, and chatbots are transforming industries at an unprecedented pace. More businesses are looking to integrate AI into their operations, but how these systems are built often gets overlooked. The main ingredient? Data. A machine learning model is only as good as the data it’s trained on. But where does all this data come from? A significant portion of the data we rely on is publicly available, often found in social media posts, user reviews, and other online content. For sentiment analysis, one of the richest sources of data is Twitter, which continuously streams real-time user-generated content. The challenge, however, lies in how to collect this data effectively and cleanly. The Problem with Traditional Twitter Data Collection When I began working on a sentiment analysis project using Twitter data, I initially considered using the Twitter API. However, I quickly encountered a significant limitation: the free-tier API only provides access to tweets from the previous seven days. For comprehensive sentiment analysis, especially for trending topics or political events spanning weeks or months, this restriction severely limits the scope and quality of the data available. While upgrading to a higher API tier was an option, the cost was prohibitive for a small-scale project like mine. This left me with two paths: attempt to collect the data manually through traditional scraping (which came with its own set of challenges like CAPTCHAs, redundancies, and incomplete records) or find an alternative solution. That's where Bright Data comes into play, solving these issues seamlessly by offering clean, scalable, and historical Twitter datasets. Instead of being confined by API limitations, I could access diverse, high-quality data for my project without the usual hassles. Step-by-Step Guide to Using Bright Data’s Prebuilt Twitter Dataset In this article, I’ll walk you through how you can leverage Bright Data’s Marketplace to acquire high-quality Twitter data for sentiment analysis and share my experience. Step 1: Sign Up and Access the Bright Data Platform Head to the Bright Data website and create a free account to start. Once logged in, you'll be directed to the Data Marketplace, where various prebuilt datasets can be accessed, including those related to Twitter. Step 2: Search for the Relevant Dataset Once you select your dataset, you can download it directly to your local machine or access it via API. The quick process allows you to start working with high-quality data almost immediately. I used the Twitter-posts data set Step 3: Load and Clean the Data The raw Twitter data typically contains a lot of metadata that may not be relevant for sentiment analysis, such as user information, image URLs, and other non-textual data. We'll focus on the text of the tweets, as that is where the sentiment analysis will be applied, along with relevant engagement metrics like reposts and likes. import pandas as pd
df = pd.read_csv("path_to_your_dataset.csv")
df.head() The next step is to clean the data by handling missing values and removing irrelevant columns. df_clean = df[['user_posted', 'description', 'date_posted', 'reposts', 'likes']]
df_clean = df_clean.dropna(subset=['description'])
df_clean = df_clean.drop_duplicates(subset=['description'])
df_clean.head() Step 4: Sentiment Analysis — Extracting Meaning from Tweets Now that we have cleaned the data and have the relevant text, it’s time to dive into the sentiment analysis. Sentiment analysis means determining the emotional tone behind words—whether the tweet is positive, negative, or neutral. I’ve used a popular library called VADER (Valence Aware Dictionary and Sentiment Reasoner), specifically designed for social media text. It’s great at handling informal language like slang and emojis. The code snippet below performs sentiment analysis using VADER from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer

analyzer = SentimentIntensityAnalyzer()

def get_sentiment(text):
    sentiment = analyzer.polarity_scores(text)
    return sentiment['compound']  

df_clean['sentiment'] = df_clean['description'].apply(get_sentiment)

df_clean.head() Step 5: Analyzing Sentiment Distribution Once sentiment scores are calculated, we can analyze the distribution to see how positive or negative the tweets are on average. This can provide insights into the overall public mood about a particular topic or event. We can also visualize the sentiment distribution using a histogram or wordcloud to identify key emotions. This plot will give you a good sense of the sentiment spread, whether it's leaning toward positive, negative, or neutral. In this case, There’s a high frequency of neutral tweets. Step 6: Correlating Sentiment with Engagement Now that we have the sentiment scores for each tweet, the next step is to explore whether there's any correlation between sentiment and engagement metrics like likes and reposts. In social media analytics, one might expect that more positive sentiment could lead to higher engagement (more likes, more reposts). However, is that truly the case? To find out, we can calculate the correlation between the sentiment scores and engagement metrics, the results are: Correlation between sentiment and likes: 0.022806738610786123 Correlation between sentiment and reposts: 0.008885789875330416 Given the weak correlation values from my data analysis, we see that there isn't a strong connection between sentiment and engagement. The correlation between sentiment and likes was 0.02, and for sentiment and reposts, it was only 0.008. These values suggest that engagement metrics like likes and reposts are not significantly influenced by the sentiment of tweets. Rethinking the Role of Sentiment in Engagement The weak correlation between sentiment and engagement shows that there are other factors at play. While sentiment analysis helps us understand public opinion, engagement is likely driven by additional factors, such as: Content relevance: Trending topics tend to generate more engagement regardless of sentiment.
User influence: Popular accounts often receive higher engagement, irrespective of their tweet’s sentiment.
Timing: Tweets posted during live events or peak times are more likely to get attention. While sentiment analysis is a valuable tool for understanding how people feel about a topic, it’s not always a reliable predictor of engagement. To gain deeper insights into user behavior, it’s essential to consider other factors such as the content’s timing, relevance, and the user’s influence. The Role of Data in Machine Learning In today's AI-driven landscape, machine learning, AI, and chatbots are transforming industries at an unprecedented pace. More businesses are looking to integrate AI into their operations, but how these systems are built often gets overlooked. The main ingredient? Data. A machine learning model is only as good as the data it’s trained on. But where does all this data come from? A significant portion of the data we rely on is publicly available, often found in social media posts, user reviews, and other online content. For sentiment analysis , one of the richest sources of data is Twitter , which continuously streams real-time user-generated content. The challenge, however, lies in how to collect this data effectively and cleanly. sentiment analysis Twitter The Problem with Traditional Twitter Data Collection When I began working on a sentiment analysis project using Twitter data, I initially considered using the Twitter API. However, I quickly encountered a significant limitation: the free-tier API only provides access to tweets from the previous seven days. For comprehensive sentiment analysis, especially for trending topics or political events spanning weeks or months, this restriction severely limits the scope and quality of the data available. While upgrading to a higher API tier was an option, the cost was prohibitive for a small-scale project like mine. This left me with two paths: attempt to collect the data manually through traditional scraping (which came with its own set of challenges like CAPTCHAs, redundancies, and incomplete records) or find an alternative solution. That's where Bright Data comes into play, solving these issues seamlessly by offering clean, scalable, and historical Twitter datasets. Instead of being confined by API limitations, I could access diverse, high-quality data for my project without the usual hassles. Step-by-Step Guide to Using Bright Data’s Prebuilt Twitter Dataset In this article, I’ll walk you through how you can leverage Bright Data’s Marketplace to acquire high-quality Twitter data for sentiment analysis and share my experience. Step 1: Sign Up and Access the Bright Data Platform Head to the Bright Data website and create a free account to start. Once logged in, you'll be directed to the Data Marketplace, where various prebuilt datasets can be accessed, including those related to Twitter. Step 2: Search for the Relevant Dataset Once you select your dataset, you can download it directly to your local machine or access it via API. The quick process allows you to start working with high-quality data almost immediately. I used the Twitter-posts data set Step 3: Load and Clean the Data The raw Twitter data typically contains a lot of metadata that may not be relevant for sentiment analysis, such as user information, image URLs, and other non-textual data. We'll focus on the text of the tweets, as that is where the sentiment analysis will be applied, along with relevant engagement metrics like reposts and likes. import pandas as pd
df = pd.read_csv("path_to_your_dataset.csv")
df.head() import pandas as pd
df = pd.read_csv("path_to_your_dataset.csv")
df.head() The next step is to clean the data by handling missing values and removing irrelevant columns. df_clean = df[['user_posted', 'description', 'date_posted', 'reposts', 'likes']]
df_clean = df_clean.dropna(subset=['description'])
df_clean = df_clean.drop_duplicates(subset=['description'])
df_clean.head() df_clean = df[['user_posted', 'description', 'date_posted', 'reposts', 'likes']]
df_clean = df_clean.dropna(subset=['description'])
df_clean = df_clean.drop_duplicates(subset=['description'])
df_clean.head() Step 4: Sentiment Analysis — Extracting Meaning from Tweets Now that we have cleaned the data and have the relevant text, it’s time to dive into the sentiment analysis . Sentiment analysis means determining the emotional tone behind words—whether the tweet is positive, negative, or neutral. sentiment analysis I’ve used a popular library called VADER (Valence Aware Dictionary and Sentiment Reasoner), specifically designed for social media text. It’s great at handling informal language like slang and emojis. VADER The code snippet below performs sentiment analysis using VADER from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer

analyzer = SentimentIntensityAnalyzer()

def get_sentiment(text):
    sentiment = analyzer.polarity_scores(text)
    return sentiment['compound']  

df_clean['sentiment'] = df_clean['description'].apply(get_sentiment)

df_clean.head() from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer

analyzer = SentimentIntensityAnalyzer()

def get_sentiment(text):
    sentiment = analyzer.polarity_scores(text)
    return sentiment['compound']  

df_clean['sentiment'] = df_clean['description'].apply(get_sentiment)

df_clean.head() Step 5: Analyzing Sentiment Distribution Once sentiment scores are calculated, we can analyze the distribution to see how positive or negative the tweets are on average. This can provide insights into the overall public mood about a particular topic or event. We can also visualize the sentiment distribution using a histogram or wordcloud to identify key emotions. histogram wordcloud This plot will give you a good sense of the sentiment spread, whether it's leaning toward positive, negative, or neutral. In this case, There’s a high frequency of neutral tweets. Step 6: Correlating Sentiment with Engagement Now that we have the sentiment scores for each tweet, the next step is to explore whether there's any correlation between sentiment and engagement metrics like likes and reposts. In social media analytics, one might expect that more positive sentiment could lead to higher engagement (more likes, more reposts). However, is that truly the case? To find out, we can calculate the correlation between the sentiment scores and engagement metrics, the results are: Correlation between sentiment and likes: 0.022806738610786123 Correlation between sentiment and likes: 0.022806738610786123 Correlation between sentiment and reposts: 0.008885789875330416 Correlation between sentiment and reposts: 0.008885789875330416 Given the weak correlation values from my data analysis, we see that there isn't a strong connection between sentiment and engagement. The correlation between sentiment and likes was 0.02, and for sentiment and reposts, it was only 0.008. These values suggest that engagement metrics like likes and reposts are not significantly influenced by the sentiment of tweets . engagement metrics like likes and reposts are not significantly influenced by the sentiment of tweets Rethinking the Role of Sentiment in Engagement The weak correlation between sentiment and engagement shows that there are other factors at play. While sentiment analysis helps us understand public opinion, engagement is likely driven by additional factors, such as: Content relevance: Trending topics tend to generate more engagement regardless of sentiment. User influence: Popular accounts often receive higher engagement, irrespective of their tweet’s sentiment. Timing: Tweets posted during live events or peak times are more likely to get attention. Content relevance : Trending topics tend to generate more engagement regardless of sentiment. Content relevance User influence : Popular accounts often receive higher engagement, irrespective of their tweet’s sentiment. User influence Timing : Tweets posted during live events or peak times are more likely to get attention. Timing While sentiment analysis is a valuable tool for understanding how people feel about a topic, it’s not always a reliable predictor of engagement. To gain deeper insights into user behavior, it’s essential to consider other factors such as the content’s timing, relevance, and the user’s influence.

How to Utilize Bright Data for Sentiment Analysis 

كيفية الاستفادة من البيانات الذكية لتحليل المشاعر

Com utilitzar dades brillants per a l'anàlisi de sentiments 

Sådan bruges lyse data til sentimentanalyse 

Cómo utilizar datos brillantes para el análisis de sentimientos

نحوه استفاده از داده های روشن برای تحلیل احساسات 

Ki jan yo itilize done ki klere pou analiz santiman 

感情分析にブライトデータを活用する方法

Мэдрэмжийн шинжилгээнд тод өгөгдлийг хэрхэн ашиглах вэ 

د احساس تحلیل لپاره روښانه ډیټا څنګه وکاروئ 

Как использовать яркие данные для анализа настроений

Sida Loo Isticmaalo Xogta Iftiiminaysa ee Falanqaynta Dareenka 

Ndlela Yo Tirhisa Data Yo Vonakala Eka Nxopaxopo Wa Mintlhaveko 

There is a popular saying that: Data is the new gold, and the web (internet) is the biggest gold mine. As a business owner, student, drop shipper, or corporate worker, you must have faced the risk of going out of business if you lack access to some (if not the most) crucial piece of data. But fear not; there is a solution that helps you unlock the full potential of web data scraping/collection, which I will show you in this article. Why Do We Collect Data: The Essence of Data Scraping/Collection Before I dive into the challenges faced with data scraping/collection and the wonders of Bright Data solutions, it is important to understand the essence of data scraping in this digital age that we are in. Data scraping is a powerful technique that efficiently navigates the internet to extract specific valuable informations, acting like a personal assistant for data retrieval. Whether you seek to analyze market trends and performance, track competitor prices, or simply satisfy your thirst for knowledge, data scraping is the key to making your life easier. Challenges With Data Scraping: The Limitations Before Bright Data Solutions Like an adventure, data scraping comes with its share of challenges and obstacles. One significant hurdle is the risk of websites feeling under attack when excessive data is scraped too quickly. To counter such attempts, websites deploy defensive measures, such as CAPTCHAs, to determine whether the user is a human or a computer. If the volume of requests persists, the website may resort to blocking users, hindering or halting data scraping altogether. To avoid such unfavorable situations, a more versatile and intelligent solution is required. Bright Data: The Ultimate Solution for Web Data Collection Bright Data comes as the ultimate solution for web data collection tailored specifically for businesses. With its automated online data collection platform, Bright Data offers unlimited access to the vast wisdom of the web without exposing you to any risks. How Bright Data Works The magic behind Bright Data lies in its web unlocker: a unique P2P network composed of millions of genuine residential and mobile IPs from across the globe. These IPs serve as secret identities, making your data requests appear as if they come from real people, making it highly unlikely for websites to block them. Why I Am Using Bright Data: The Power of Using Bright Data 1. Versatility and Ease of Use Bright Data shines with its user-friendly approach. To start scraping vast amounts of data, all you need is a scraping browser — an automated browser that allows you to interact with websites just like a human user. It efficiently handles challenges such as CAPTCHAs, fingerprints, and retries while mimicking the behavior of a regular user. Compatible with popular automation tools like Puppeteer and Playwright and programming languages like Python and Node.js, the scraping browser makes data collection seamless and efficient. 2. Ready-to-Use Data Sets If you ever need ready-to-use data, Bright Data’s data sets tool provides pre-collected and structured data from various domains, including popular platforms like LinkedIn, Amazon, Crunchbase, Indeed, and Twitter. Regularly updated, these comprehensive data sets offer you a treasure trove of information at your fingertips. Practical Examples of What I Have Used Bright Data for 1. E-Commerce: Monitoring Competitor Prices and Enriching Product Catalogs In the world of e-commerce, web data plays a crucial role in monitoring competitor prices, ensuring product availability, and understanding customer reviews. For instance, a global Beauty and Personal Care Marketplace efficiently uses web data to check competitor prices in real-time, enabling them to adjust pricing strategies and optimize promotions on the fly, enhancing their market competitiveness. 2. Social Media: Analyzing Consumer Sentiment and Trends Social media platforms offer a gold mine of user-generated content, providing valuable insights into consumer behavior, trends, and sentiment. A global digital marketing agency effectively analyzes consumer sentiment towards their client’s brands using web data from various social media platforms. This deeper understanding empowers them to refine marketing strategies and achieve better results. In Conclusion: The Bright Future of Data Collection As I conclude with Bright Data's powerful web data collection solutions, one thing becomes very clear; data scraping is a game-changer in today’s digital landscape. Whether you are a business owner, entrepreneur, or data professional, harnessing the power of web data can unlock countless opportunities for growth and success. Traditional manual data collection methods are slow, labor-intensive, and error-prone, rendering them ineffective and inefficient in the face of today’s data-driven world. By automating the process with Bright Data, you can extract specific and relevant information from web pages effortlessly, propelling your business to new heights. So, if you find yourself in need of vast amounts of data, fast and easy, look no further than Bright Data. Embrace the power of web data, and let it guide you to new horizons of success in this ever-evolving digital landscape. There is a popular saying that: Data is the new gold, and the web (internet) is the biggest gold mine. Data is the new gold, and the web (internet) is the biggest gold mine. As a business owner, student, drop shipper, or corporate worker, you must have faced the risk of going out of business if you lack access to some (if not the most) crucial piece of data. But fear not; there is a solution that helps you unlock the full potential of web data scraping/collection, which I will show you in this article. (if not the most) Why Do We Collect Data: The Essence of Data Scraping/Collection Before I dive into the challenges faced with data scraping/collection and the wonders of Bright Data solutions, it is important to understand the essence of data scraping in this digital age that we are in. Bright Data Data scraping is a powerful technique that efficiently navigates the internet to extract specific valuable informations, acting like a personal assistant for data retrieval. Whether you seek to analyze market trends and performance, track competitor prices, or simply satisfy your thirst for knowledge, data scraping is the key to making your life easier. Challenges With Data Scraping: The Limitations Before Bright Data Solutions Bright Data Like an adventure, data scraping comes with its share of challenges and obstacles. One significant hurdle is the risk of websites feeling under attack when excessive data is scraped too quickly. To counter such attempts, websites deploy defensive measures, such as CAPTCHAs, to determine whether the user is a human or a computer. If the volume of requests persists, the website may resort to blocking users, hindering or halting data scraping altogether. To avoid such unfavorable situations, a more versatile and intelligent solution is required. Bright Data: The Ultimate Solution for Web Data Collection Bright Data comes as the ultimate solution for web data collection tailored specifically for businesses. With its automated online data collection platform, Bright Data offers unlimited access to the vast wisdom of the web without exposing you to any risks. Bright Data How Bright Data Works How Bright Data Works The magic behind Bright Data lies in its web unlocker: a unique P2P network composed of millions of genuine residential and mobile IPs from across the globe. These IPs serve as secret identities, making your data requests appear as if they come from real people, making it highly unlikely for websites to block them. Why I Am Using Bright Data: The Power of Using Bright Data 1. Versatility and Ease of Use 1. Versatility and Ease of Use Bright Data shines with its user-friendly approach. To start scraping vast amounts of data, all you need is a scraping browser — an automated browser that allows you to interact with websites just like a human user. It efficiently handles challenges such as CAPTCHAs, fingerprints, and retries while mimicking the behavior of a regular user. Compatible with popular automation tools like Puppeteer and Playwright and programming languages like Python and Node.js, the scraping browser makes data collection seamless and efficient. 2. Ready-to-Use Data Sets 2. Ready-to-Use Data Sets If you ever need ready-to-use data, Bright Data’s data sets tool provides pre-collected and structured data from various domains, including popular platforms like LinkedIn, Amazon, Crunchbase, Indeed, and Twitter. Regularly updated, these comprehensive data sets offer you a treasure trove of information at your fingertips. Practical Examples of What I Have Used Bright Data for 1. E-Commerce: Monitoring Competitor Prices and Enriching Product Catalogs 1. E-Commerce: Monitoring Competitor Prices and Enriching Product Catalogs In the world of e-commerce, web data plays a crucial role in monitoring competitor prices, ensuring product availability, and understanding customer reviews. For instance, a global Beauty and Personal Care Marketplace efficiently uses web data to check competitor prices in real-time, enabling them to adjust pricing strategies and optimize promotions on the fly, enhancing their market competitiveness. 2. Social Media: Analyzing Consumer Sentiment and Trends 2. Social Media: Analyzing Consumer Sentiment and Trends Social media platforms offer a gold mine of user-generated content, providing valuable insights into consumer behavior, trends, and sentiment. A global digital marketing agency effectively analyzes consumer sentiment towards their client’s brands using web data from various social media platforms. This deeper understanding empowers them to refine marketing strategies and achieve better results. In Conclusion: The Bright Future of Data Collection As I conclude with Bright Data's powerful web data collection solutions, one thing becomes very clear; data scraping is a game-changer in today’s digital landscape. Whether you are a business owner, entrepreneur, or data professional, harnessing the power of web data can unlock countless opportunities for growth and success. Traditional manual data collection methods are slow, labor-intensive, and error-prone, rendering them ineffective and inefficient in the face of today’s data-driven world. By automating the process with Bright Data, you can extract specific and relevant information from web pages effortlessly, propelling your business to new heights. So, if you find yourself in need of vast amounts of data, fast and easy, look no further than Bright Data. Embrace the power of web data, and let it guide you to new horizons of success in this ever-evolving digital landscape.

How Bright Data Simplifies Web Scraping/Data Collection for AI Training

Як AI Bright Data зрабіў скрабаванне/збор вэб-даных простым: праблемы перад рашэннямі AI Bright Data

Com Bright Data AI va fer que el raspat/recollida de dades web sigui sense esforç: els reptes abans de les solucions Bright Data AI

Cómo Bright Data AI hizo que la recopilación y el raspado de datos web fueran sencillos: los desafíos que enfrentaban las soluciones de Bright Data AI

Kako je Bright Data AI skrapiranje/prikupljanje web podataka učinio jednostavnim: Izazovi pred Bright Data AI rješenjima

Bright Data AI が Web データのスクレイピング/収集をいかに簡単にしたか: Bright Data AI ソリューションの課題

Cara AI Data Cerah Menjadikan Pengikisan/Pengumpulan Data Web Mudah: Cabaran Sebelum Penyelesaian AI Data Cerah

څنګه روښانه ډیټا AI د ویب ډیټا سکریپینګ/راټولول بې ځایه کړي: د روښانه ډیټا AI حلونو دمخه ننګونې

Bright Data AI විසින් Web Data Scraping/එකතු කිරීම වෙහෙසකින් තොරව සිදු කළේ කෙසේද: Bright Data AI විසඳුම් ඉදිරියේ ඇති අභියෝග

Jinsi Data Angavu AI Ilivyofanya Uchakachuaji/Ukusanyaji wa Data ya Wavuti Bila Jitihada: Changamoto Kabla ya Suluhisho Mkali za Data AI

எப்படி பிரைட் டேட்டா AI இணையத் தரவை ஸ்கிராப்பிங்/சேகரிப்பதை சிரமமின்றி உருவாக்கியது: பிரைட் டேட்டா AI தீர்வுகளுக்கு முன் உள்ள சவால்கள்

کس طرح برائٹ ڈیٹا اے آئی نے ویب ڈیٹا کو اسکریپنگ/جمع کرنے کو آسان بنایا: برائٹ ڈیٹا اے آئی کے حل کے سامنے چیلنجز

Bright Data AI đã giúp việc thu thập/sao chép dữ liệu web trở nên dễ dàng như thế nào: Những thách thức trước khi có giải pháp Bright Data AI

The rapid growth of AI is transforming many areas of life, including education, with ChatGPT at the forefront of this change. Recently, OpenAI, the company behind ChatGPT, introduced ChatGPT Edu—a tailored version of its flagship AI tool built for universities to “responsibly” deploy AI to students, faculty, researchers, and campus operations. Riding on the success of its ChatGPT Edu in Universities like the University of Oxford, Wharton School of the University of Pennsylvania(opens in a new window), University of Texas at Austin, Arizona State University(opens in a new window), and Columbia University in the City of New York, ChatGPT edu comes with a lot of promise— one that can help universities integrate AI into every aspect concerning students and faculties. But as always, with every ease comes hardship, and ChatGPT Edu too can be a double-edged sword. But we’ll discuss the downsides a little later. Let’s lead with how ChatGPT Edu can be constructive for higher educational institutions, their students, and faculties. Beginning with the fact that ChatGPT leverages AI, it shouldn’t be a debate that its Edu version can provide faculty members and students with a satisfying level of personalization. By tailoring content and study material as per every student’s need, it can be the silver bullet that universities desperately need. With faculties and mentors grappling with the grueling task of managing the rising number of students in a class, personalization through AI can bring tremendous results for them. The fact that it can identify knowledge gaps and adapt to different learning styles symbolizes hope for the future of academia. AI has always attracted keen minds and ChatGPT Edu, given its inherent character to integrate AI with interactive activities, can make learning more engaging and interactive. Students who are disengaged from routine lectures and other pedagogical processes may find AI’s use in education to be an attractive prospect. This can help keep the students more engaged– a challenge that is commonly observed among students. They are highly disoriented, given the fact that a vast majority of higher educational institutions are still sticking to rote learning practices. Enter ChatGPT Edu, by engaging students in simulated discussions, role-playing scenarios, and critical thinking skills, it can help them stay focused and better oriented, and help suitably streamline their academic priorities. For students with learning disabilities, ChatGPT Edu can be the holy grail. It can offer personalized feedback in various formats, including audio or visual aids, catering to different learning preferences. ChatGPT Edu can also help universities stay clear of the language barriers that often arise when students and faculties of one country inhabit the other. Offering real-time language translation and support can allow students to participate more actively in classroom discussions and access course materials in their preferred language. What’s beyond this is it can also assist in personalized pronunciation practice and real-time feedback, fostering greater confidence and fluency for non-native speakers. The faculty members, moreover, can use the translation feature to ensure all students understand the course material the way they want them to. They can translate lectures, presentations, and handouts, allowing students to follow along and engage more effectively by harnessing the tool properly. We’ll have to see how ChatGPT’s Edu version performs once it is widely circulated in the public domain. For students from diverse backgrounds who face difficulty understanding concepts and things, it can be a big breather as it can translate complex concepts and methods into simpler language. Being able to offer text-to-speech conversion, and the capability to generate visual aids and summaries will surely be a cherry on the cake for both students and faculties alike. Lastly, what seems to be interesting from the faculty’s perspective is that it can offer continued assistance to faculty research. It can certainly help analyze vast amounts of data, generate research hypotheses, and streamline literature reviews, freeing up valuable time for more focused research endeavors. AI in education isn’t a capricious idea anymore. The downsides One of the biggest concerns arising out of the use of AI in educational institutions is there is a chance of students getting too reliant on it for everything. This can have serious repercussions. They can lose touch with realistic teaching pedagogy and become more AI-dependent thereby stepping into that alarming “virtual” zone of learning, where anything without the aid of AI is less understood or best, misunderstood. An equally concerning aspect is the fact it can severely hinder critical thinking and problem-solving skills for a lot of the students. Secondly, it’s believed AI is only as good as the data it's trained on. If that data is biased or fed with wrong inputs, ChatGPT Edu could perpetuate stereotypes or deliver inaccurate information—something which can lead the learner astray, thereby inhibiting the whole learning experience and marring the learning curve. Lastly, overuse of this technology could blur the lines between original and duplicate content (read plagiarism). Rightly so! Plagiarism has been a serious concern for academic institutions, as cases of AI-blended learning have risen over the years. And these concerns of AI and plagiarism are legitly myriad. We’ll have to see how this tailor-made ChatGPT tool can minimize incidents of students and learners plagiarizing content in universities. Summing Up Education offers incredible potential for the application of AI technologies. With the introduction of ChatGPT Edu and its subsequent integration into educational institutions across all research and learning touchpoints, it seems the educational landscape may be in for some unanticipated and enormous change. Nevertheless, learning the tricks of AI-based learning practices and dupery that could be possibly built around it by mala fide learners, can certainly be a solution and shouldn’t be taken for granted. While on one hand, adopting regular audits and data curation practices can help minimize bias and misinformation, on the other, universities can design courses that blend ChatGPT Edu for personalized learning while prioritizing interactive activities and discussions that are likely to produce the maximum output while remaining efficient. With the integration of this innovative tool, learning practices bear a likelihood of showing improved outcomes but it also brings with itself a multitude of vulnerabilities and risks. It just remains to be seen if post its integration with universities on a massive scale, whether it will endanger institutions and recede learning to a dangerous low or will it push towards an academic environment that appears technologically and educationally utopian. The rapid growth of AI is transforming many areas of life, including education, with ChatGPT at the forefront of this change. Recently, OpenAI, the company behind ChatGPT, introduced ChatGPT Edu—a tailored version of its flagship AI tool built for universities to “responsibly” deploy AI to students, faculty, researchers, and campus operations. Riding on the success of its ChatGPT Edu in Universities like the University of Oxford, Wharton School of the University of Pennsylvania(opens in a new window), University of Texas at Austin, Arizona State University(opens in a new window), and Columbia University in the City of New York, ChatGPT edu comes with a lot of promise— one that can help universities integrate AI into every aspect concerning students and faculties. But as always, with every ease comes hardship, and ChatGPT Edu too can be a double-edged sword. But we’ll discuss the downsides a little later. the success of its ChatGPT Edu in Universities Let’s lead with how ChatGPT Edu can be constructive for higher educational institutions, their students, and faculties. Beginning with the fact that ChatGPT leverages AI , it shouldn’t be a debate that its Edu version can provide faculty members and students with a satisfying level of personalization. By tailoring content and study material as per every student’s need, it can be the silver bullet that universities desperately need. With faculties and mentors grappling with the grueling task of managing the rising number of students in a class, personalization through AI can bring tremendous results for them. The fact that it can identify knowledge gaps and adapt to different learning styles symbolizes hope for the future of academia. ChatGPT leverages AI ChatGPT leverages AI personalization through AI personalization through AI AI has always attracted keen minds and ChatGPT Edu, given its inherent character to integrate AI with interactive activities, can make learning more engaging and interactive. Students who are disengaged from routine lectures and other pedagogical processes may find AI’s use in education to be an attractive prospect. This can help keep the students more engaged– a challenge that is commonly observed among students. They are highly disoriented, given the fact that a vast majority of higher educational institutions are still sticking to rote learning practices. Enter ChatGPT Edu, by engaging students in simulated discussions, role-playing scenarios, and critical thinking skills, it can help them stay focused and better oriented, and help suitably streamline their academic priorities. For students with learning disabilities, ChatGPT Edu can be the holy grail. It can offer personalized feedback in various formats, including audio or visual aids, catering to different learning preferences. ChatGPT Edu can also help universities stay clear of the language barriers that often arise when students and faculties of one country inhabit the other. Offering real-time language translation and support can allow students to participate more actively in classroom discussions and access course materials in their preferred language. What’s beyond this is it can also assist in personalized pronunciation practice and real-time feedback, fostering greater confidence and fluency for non-native speakers. The faculty members, moreover, can use the translation feature to ensure all students understand the course material the way they want them to. They can translate lectures, presentations, and handouts, allowing students to follow along and engage more effectively by harnessing the tool properly. We’ll have to see how ChatGPT’s Edu version performs once it is widely circulated in the public domain. translation feature translation feature For students from diverse backgrounds who face difficulty understanding concepts and things, it can be a big breather as it can translate complex concepts and methods into simpler language. Being able to offer text-to-speech conversion, and the capability to generate visual aids and summaries will surely be a cherry on the cake for both students and faculties alike. Lastly, what seems to be interesting from the faculty’s perspective is that it can offer continued assistance to faculty research. It can certainly help analyze vast amounts of data, generate research hypotheses, and streamline literature reviews, freeing up valuable time for more focused research endeavors. AI in education isn’t a capricious idea anymore. The downsides One of the biggest concerns arising out of the use of AI in educational institutions is there is a chance of students getting too reliant on it for everything. This can have serious repercussions. They can lose touch with realistic teaching pedagogy and become more AI-dependent thereby stepping into that alarming “virtual” zone of learning, where anything without the aid of AI is less understood or best, misunderstood. An equally concerning aspect is the fact it can severely hinder critical thinking and problem-solving skills for a lot of the students. Secondly, it’s believed AI is only as good as the data it's trained on. If that data is biased or fed with wrong inputs, ChatGPT Edu could perpetuate stereotypes or deliver inaccurate information—something which can lead the learner astray, thereby inhibiting the whole learning experience and marring the learning curve. Lastly, overuse of this technology could blur the lines between original and duplicate content (read plagiarism). Rightly so! Plagiarism has been a serious concern for academic institutions, as cases of AI-blended learning have risen over the years. And these concerns of AI and plagiarism are legitly myriad. We’ll have to see how this tailor-made ChatGPT tool can minimize incidents of students and learners plagiarizing content in universities. AI and plagiarism AI and plagiarism Summing Up Education offers incredible potential for the application of AI technologies. With the introduction of ChatGPT Edu and its subsequent integration into educational institutions across all research and learning touchpoints, it seems the educational landscape may be in for some unanticipated and enormous change. incredible potential for the application incredible potential for the application Nevertheless, learning the tricks of AI-based learning practices and dupery that could be possibly built around it by mala fide learners, can certainly be a solution and shouldn’t be taken for granted. While on one hand, adopting regular audits and data curation practices can help minimize bias and misinformation, on the other, universities can design courses that blend ChatGPT Edu for personalized learning while prioritizing interactive activities and discussions that are likely to produce the maximum output while remaining efficient. With the integration of this innovative tool, learning practices bear a likelihood of showing improved outcomes but it also brings with itself a multitude of vulnerabilities and risks. It just remains to be seen if post its integration with universities on a massive scale, whether it will endanger institutions and recede learning to a dangerous low or will it push towards an academic environment that appears technologically and educationally utopian.

ChatGPT Edu: Dawn of a Brighter Academic Future, or a Slippery Slope Towards Diminished Learning?

Data is the lifeblood of AI. And as the world embraces the potential of intelligent systems, the value of the data that fuels them becomes even clearer. To foster meaningful conversations around data collection for AI and LLMs, Bright Data and HackerNoon are delighted to launch the AI Writing Contest. Share your thoughts, research, and/or projects on HackerNoon using the #ai tag for a chance to win from a $2500 prize pool. What to Write About Demonstrate how you've used Bright Data to collect web data for AI, and provide a detailed analysis of the tasks performed and their results.
Discuss new trends in AI and LLM data collection and their potential impact on current practices.
Highlight web scraping tools that could transform AI training data.
Demonstrate how public web data can be labeled for AI training, with examples of how labeled data improved AI performance in specific applications.
Explore how Bright Data's tools can be leveraged for improved AI and LLM training.
Provide examples of real-world applications where data collected at scale through web scraping has been used to train successful AI models.
Examine techniques for optimizing data pipelines to enhance the efficiency and effectiveness of training large language models.
Discuss how Bright Data's APIs and Marketplace provide sources for text, video, images, audio, and other dataset types needed for AI model training, and how they can be used for various AI tasks. See the full list of writing prompts here. Get Started with Bright Data: To help you prepare for the contest, Bright Data is offering $15 in credit for you to test out its tools. Use this credit to explore:


Web Data for AI


Dataset Marketplace


Participants will have access to free datasets, highlighted on the platform. For specific dataset requests, contact hackernoon@brightdata.com, and the data will be provided to your account within 24 hours. Prize Breakdown for the AI Writing Contest

$$$



Best Bright Data Story

$1000



Best AI Story

$1000



Best Web Scraping Story

$500 The AI Writing Contest Rules & Guidelines Must be 18+ to enter (no location restrictions)
Create a HackerNoon account
You can enter the contest by adding the #ai tag to your story.
No AI-generated entries. (Read: the acceptable way to use AI on HackerNoon) FAQs Can I Write Under a Pen Name? Yes!You can use your real name, or a pseudonym when setting up your HackerNoon profile. How Long Will the Contest Run? The Contest will run for 3 months, starting on September 1, 2024, and ending on December 1, 2024. How are the winners selected? After 3 months, we’ll share a shortlist of the stories that receive the most eyeballs (real humans, not bots!).
Next, the shortlisted stories will be voted on by HackerNoon staff.
Then, the best in each prize category; AI, Bright Data, and Web Scraping, will be selected and announced Ready to give it a shot? Start a draft now, or use this template to enter the AI writing contest. Good luck and may the force be with you! Data is the lifeblood of AI. And as the world embraces the potential of intelligent systems, the value of the data that fuels them becomes even clearer. Data is the lifeblood of AI. To foster meaningful conversations around data collection for AI and LLMs, Bright Data and HackerNoon are delighted to launch the AI Writing Contest . AI Writing Contest Share your thoughts, research, and/or projects on HackerNoon using the #ai tag for a chance to win from a $2500 prize pool. Share your thoughts, research, and/or projects on HackerNoon using the #ai tag for a chance to win from a $2500 prize pool. Share your thoughts, research, and/or projects on HackerNoon using the #ai tag for a chance to win from a $2500 prize pool. #ai What to Write About Demonstrate how you've used Bright Data to collect web data for AI, and provide a detailed analysis of the tasks performed and their results. Discuss new trends in AI and LLM data collection and their potential impact on current practices. Highlight web scraping tools that could transform AI training data. Demonstrate how public web data can be labeled for AI training, with examples of how labeled data improved AI performance in specific applications. Explore how Bright Data's tools can be leveraged for improved AI and LLM training. Provide examples of real-world applications where data collected at scale through web scraping has been used to train successful AI models. Examine techniques for optimizing data pipelines to enhance the efficiency and effectiveness of training large language models. Discuss how Bright Data's APIs and Marketplace provide sources for text, video, images, audio, and other dataset types needed for AI model training, and how they can be used for various AI tasks. Demonstrate how you've used Bright Data to collect web data for AI, and provide a detailed analysis of the tasks performed and their results. Discuss new trends in AI and LLM data collection and their potential impact on current practices. Highlight web scraping tools that could transform AI training data. Demonstrate how public web data can be labeled for AI training, with examples of how labeled data improved AI performance in specific applications. Explore how Bright Data's tools can be leveraged for improved AI and LLM training. Provide examples of real-world applications where data collected at scale through web scraping has been used to train successful AI models. Examine techniques for optimizing data pipelines to enhance the efficiency and effectiveness of training large language models. Discuss how Bright Data's APIs and Marketplace provide sources for text, video, images, audio, and other dataset types needed for AI model training, and how they can be used for various AI tasks. See the full list of writing prompts here. See the full list of writing prompts here. See the full list of writing prompts here. here. Get Started with Bright Data: To help you prepare for the contest, Bright Data is offering $15 in credit for you to test out its tools. Use this credit to explore: Web Data for AI


Dataset Marketplace Participants will have access to free datasets, highlighted on the platform. For specific dataset requests, contact hackernoon@brightdata.com, and the data will be provided to your account within 24 hours. Get Started with Bright Data: To help you prepare for the contest, Bright Data is offering $15 in credit for you to test out its tools. Use this credit to explore: Get Started with Bright Data: Use this credit to explore: Web Data for AI Dataset Marketplace Web Data for AI Web Data for AI Web Data for AI Dataset Marketplace Dataset Marketplace Dataset Marketplace Participants will have access to free datasets, highlighted on the platform. For specific dataset requests, contact hackernoon@brightdata.com, and the data will be provided to your account within 24 hours. Participants will have access to free datasets, highlighted on the platform. For specific dataset requests, contact hackernoon@brightdata.com , and the data will be provided to your account within 24 hours. hackernoon@brightdata.com hackernoon@brightdata.com Prize Breakdown for the AI Writing Contest

$$$



Best Bright Data Story

$1000



Best AI Story

$1000



Best Web Scraping Story

$500 Prize Breakdown for the AI Writing Contest

$$$



Best Bright Data Story

$1000



Best AI Story

$1000



Best Web Scraping Story

$500 Prize Breakdown for the AI Writing Contest

$$$ Best Bright Data Story

$1000 Best AI Story

$1000 Best Web Scraping Story

$500 Prize Breakdown for the AI Writing Contest $$$ Prize Breakdown for the AI Writing Contest Prize Breakdown for the AI Writing Contest Prize Breakdown for the AI Writing Contest $$$ $$$ $$$ Best Bright Data Story $1000 Best Bright Data Story Best Bright Data Story $1000 $1000 Best AI Story $1000 Best AI Story Best AI Story $1000 $1000 Best Web Scraping Story $500 Best Web Scraping Story Best Web Scraping Story $500 $500 The AI Writing Contest Rules & Guidelines Must be 18+ to enter (no location restrictions) Create a HackerNoon account You can enter the contest by adding the #ai tag to your story. No AI-generated entries. (Read: the acceptable way to use AI on HackerNoon) Must be 18+ to enter (no location restrictions) Create a HackerNoon account Create a HackerNoon account Create a HackerNoon account You can enter the contest by adding the #ai tag to your story. #ai #ai No AI-generated entries. (Read: the acceptable way to use AI on HackerNoon ) the acceptable way to use AI on HackerNoon the acceptable way to use AI on HackerNoon FAQs Can I Write Under a Pen Name? Yes! You can use your real name, or a pseudonym when setting up your HackerNoon profile. How Long Will the Contest Run? The Contest will run for 3 months, starting on September 1, 2024, and ending on December 1, 2024. How are the winners selected? After 3 months, we’ll share a shortlist of the stories that receive the most eyeballs (real humans, not bots!). Next, the shortlisted stories will be voted on by HackerNoon staff. Then, the best in each prize category; AI, Bright Data, and Web Scraping, will be selected and announced After 3 months, we’ll share a shortlist of the stories that receive the most eyeballs (real humans, not bots!). Next, the shortlisted stories will be voted on by HackerNoon staff. Then, the best in each prize category; AI, Bright Data, and Web Scraping, will be selected and announced Ready to give it a shot? Ready to give it a shot? Start a draft now, or use this template to enter the AI writing contest. Start a draft now , or use this template to enter the AI writing contest . Start a draft now this template AI writing contest Good luck and may the force be with you! Good luck and may the force be with you!

Win Up to $2500 in the AI Writing Contest by Bright Data and HackerNoon 

ব্রাইট ডেটা এবং হ্যাকারনুন দ্বারা এআই লেখার প্রতিযোগিতায় $2500 পর্যন্ত জিতে নিন 

Gewinnen Sie bis zu 2500 $ im AI Writing Contest von Bright Data und HackerNoon

Gane hasta $2500 en el concurso de redacción sobre inteligencia artificial de Bright Data y HackerNoon

Gagnez jusqu'à 2 500 $ dans le concours d'écriture sur l'IA organisé par Bright Data et HackerNoon

ब्राइट डेटा और हैकरनून द्वारा आयोजित एआई लेखन प्रतियोगिता में $2500 तक जीतें

Bright Data と HackerNoon による AI ライティング コンテストで最大 2,500 ドルを獲得

Bright Data와 HackerNoon의 AI Writing Contest에서 최대 $2500을 획득하세요

Ganhe até $ 2.500 no concurso de redação de IA da Bright Data e HackerNoon

Выиграйте до 2500 долларов в конкурсе по написанию текстов на тему ИИ от Bright Data и HackerNoon

Bright Data ve HackerNoon'un AI Yazma Yarışmasında 2500$'a Kadar Kazanın

Thắng giải thưởng lên đến 2500 đô la trong Cuộc thi viết về AI do Bright Data và HackerNoon tổ chức

在 Bright Data 和 HackerNoon 举办的 AI 写作比赛中赢取高达 2500 美元的奖金

Bright Data infrastructure is championed around collecting only publicly available data, backed by an industry-leading know-your-customer (KYC) process and a transparent, acceptable use policy. Rated number one by customers on G2, Bright Data provides two scraping solutions, the Web Scraper API and the Scraping Browser, that simplify the data collection process. This guide covers the following: Advantages of using Bright Data’s scraping tools over traditional methods
Practical demonstration of building a dataset with a scraping browser
Overview of Bright Data’s dataset marketplace for ease of use Let’s get started. Advantages of using Bright Data’s scraping tools The Scraping Browser is one of the industry’s most cutting-edge scraping tools. With this tool, you can run your scripts on fully hosted browsers equipped with a CAPTCHA auto-solver, unlimited scalability, and residential IPs to enhance data collection. According to Bright Data CEO Or Lenchner, the Internet is the world’s largest database; the only issue is organizing its data. Using the Scraping Browser, Bright Data helps you manage your data with its in-built unblocking and hosting properties. Some of the advantages of using Bright Data’s scraping tool includes: Reason #1: Efficiency Bright Data comes embedded with a developer-first dynamic scraping ability with pre-written scripts with different scraping technologies, making it possible to accelerate the data collection process compared to traditional methods. Reason #2: Reliability Data companies trust Bright Data tools for their robustness and stability in handling large-scale data collection across their vast residential IP pool and network, ensuring high-quality and accurate data. Another critical point to note is that Bright Data offers support and resolves issues quickly for its customers around the clock. Reason #3: Global adoption Bright Data powers many global brands, which is interesting to note with Bright Insights. Bright Insights leverages deep technology infrastructure to transform public data into actionable insights that serve more than 20,000+ businesses with crucial public web data. Some notable companies that use Bright Data are Microsoft, Epson, Mozilla, and so on. One widespread use case where companies use Bright Data for their data collection needs is using data for AI. In such a scenario, Bright Data assures its customers and users (data scientists) that they always have AI training data for their machine learning models, providing everything you need from discovering, curating, and collecting web data at scale. The four essential components of the data from Bright Data are: Continuously refreshed


Clean and validated


Compliant and ethical


Scalable and performant Check out the different data types used in AI models to learn more about using datasets to train AI models and LLMs. Practical demonstration of building a dataset with a Scraping Browser According to Bright Data documentation, the Scraping Browser is one of their proxy-unlocking solutions designed to help you focus on your multi-step data collection from browsers while taking care of your full proxy and unblocking infrastructure. Some of the benefits of the Scraping Browser are: Boost developer productivity
Cut infrastructure overheads
Increase success rates
Ease of use and integration with libraries like Puppeteer, Playwright, and Selenium To get started, follow the instructions in the documentation that describes how to use the Scraping Browser. Sign up for free and receive a $5 credit. For a more profound practice on integrating and using Playwright with Python, this guide takes you through scraping public web data. Overview of Bright Data’s dataset marketplace Get fresh datasets from the Bright Data dataset marketplace, a repository of datasets from popular public websites. Browse the dataset category and select the one you want to use. The dataset pricing model is calculated based on the number of records used, whether one-time, biannual, quarterly, or monthly. Advantages of using the marketplace No-code web scraping
Strict validation methods
API for on-demand data Conclusion In this article, we discussed the importance of the Scraping Browser and the readily available pre-built datasets in the marketplace. Before scraping or understanding data from a company dataset, you need not be technical. Streamlining data serves as a means to support companies needing extensive training data and capitalizing on building efficient models. Finally, according to Forbes, the future of web scraping is intricately tied to ML and AI technologies. In 2024, scraping tools will become more intelligent, and the need for manual intervention will diminish. The implications of this can raise concerns about data privacy and ethical use if it scrapes sensitive information from the target site without consent. Are you a developer who wants to try Bright Data and its integration? It supports JavaScript, Node, Python, and programming languages like C#, Java, Go, and Ruby. Learn more Web Scraper APIs Bright Data infrastructure is championed around collecting only publicly available data, backed by an industry-leading know-your-customer (KYC) process and a transparent, acceptable use policy. Rated number one by customers on G2 , Bright Data provides two scraping solutions, the Web Scraper API and the Scraping Browser , that simplify the data collection process. by customers on G2 by customers on G2 Web Scraper API Web Scraper API Scraping Browser Scraping Browser This guide covers the following: Advantages of using Bright Data’s scraping tools over traditional methods Practical demonstration of building a dataset with a scraping browser Overview of Bright Data’s dataset marketplace for ease of use Advantages of using Bright Data’s scraping tools over traditional methods Practical demonstration of building a dataset with a scraping browser Overview of Bright Data’s dataset marketplace for ease of use Let’s get started. Advantages of using Bright Data’s scraping tools Advantages of using Bright Data’s scraping tools The Scraping Browser is one of the industry’s most cutting-edge scraping tools. With this tool, you can run your scripts on fully hosted browsers equipped with a CAPTCHA auto-solver, unlimited scalability, and residential IPs to enhance data collection. According to Bright Data CEO Or Lenchner, the Internet is the world’s largest database; the only issue is organizing its data. Using the Scraping Browser, Bright Data helps you manage your data with its in-built unblocking and hosting properties. Some of the advantages of using Bright Data’s scraping tool includes: Reason #1: Efficiency Reason #1: Efficiency Bright Data comes embedded with a developer-first dynamic scraping ability with pre-written scripts with different scraping technologies, making it possible to accelerate the data collection process compared to traditional methods. Reason #2: Reliability Reason #2: Reliability Data companies trust Bright Data tools for their robustness and stability in handling large-scale data collection across their vast residential IP pool and network, ensuring high-quality and accurate data. Another critical point to note is that Bright Data offers support and resolves issues quickly for its customers around the clock. Reason #3: Global adoption Reason #3: Global adoption Bright Data powers many global brands, which is interesting to note with Bright Insights. Bright Insights leverages deep technology infrastructure to transform public data into actionable insights that serve more than 20,000+ businesses with crucial public web data. Some notable companies that use Bright Data are Microsoft, Epson, Mozilla, and so on. One widespread use case where companies use Bright Data for their data collection needs is using data for AI. In such a scenario, Bright Data assures its customers and users (data scientists) that they always have AI training data for their machine learning models, providing everything you need from discovering, curating, and collecting web data at scale. The four essential components of the data from Bright Data are: Continuously refreshed Clean and validated Compliant and ethical Scalable and performant Continuously refreshed Continuously refreshed Clean and validated Clean and validated Compliant and ethical Compliant and ethical Scalable and performant Scalable and performant Check out the different data types used in AI models to learn more about using datasets to train AI models and LLMs. data types used in AI models data types used in AI models Practical demonstration of building a dataset with a Scraping Browser Practical demonstration of building a dataset with a Scraping Browser According to Bright Data documentation, the Scraping Browser is one of their proxy-unlocking solutions designed to help you focus on your multi-step data collection from browsers while taking care of your full proxy and unblocking infrastructure. Some of the benefits of the Scraping Browser are: Boost developer productivity Cut infrastructure overheads Increase success rates Ease of use and integration with libraries like Puppeteer, Playwright, and Selenium Boost developer productivity Cut infrastructure overheads Increase success rates Ease of use and integration with libraries like Puppeteer, Playwright, and Selenium To get started, follow the instructions in the documentation that describes how to use the Scraping Browser. Sign up for free and receive a $5 credit. documentation documentation Sign up for free Sign up for free $5 For a more profound practice on integrating and using Playwright with Python, this guide takes you through scraping public web data. this guide this guide Overview of Bright Data’s dataset marketplace Overview of Bright Data’s dataset marketplace Get fresh datasets from the Bright Data dataset marketplace , a repository of datasets from popular public websites. Browse the dataset category and select the one you want to use. Bright Data dataset marketplace Bright Data dataset marketplace The dataset pricing model is calculated based on the number of records used, whether one-time, biannual, quarterly, or monthly. Advantages of using the marketplace Advantages of using the marketplace No-code web scraping Strict validation methods API for on-demand data No-code web scraping Strict validation methods API for on-demand data Conclusion Conclusion In this article, we discussed the importance of the Scraping Browser and the readily available pre-built datasets in the marketplace. Before scraping or understanding data from a company dataset, you need not be technical. Streamlining data serves as a means to support companies needing extensive training data and capitalizing on building efficient models. Finally, according to Forbes , the future of web scraping is intricately tied to ML and AI technologies. In 2024, scraping tools will become more intelligent, and the need for manual intervention will diminish. The implications of this can raise concerns about data privacy and ethical use if it scrapes sensitive information from the target site without consent. according to Forbes according to Forbes Are you a developer who wants to try Bright Data and its integration ? It supports JavaScript, Node, Python, and programming languages like C#, Java, Go, and Ruby. try Bright Data and its integration try Bright Data and its integration Learn more Web Scraper APIs Web Scraper APIs Web Scraper APIs Web Scraper APIs

Streamlining AI Data Collection with Bright Data’s Scraping Browser

The Noonification: Meet Bright Data: HackerNoon Company of the Week (7/22/2024)

Another week, another Company of the Week! Every week, we share an awesome tech brand from our tech company database, making their evergreen mark on the internet. This unique HackerNoon database ranks S&P 500 companies and top startups of the year alike. This week, we are proud to present Bright Data - one of the largest data collection platform on the planet internet having more than 15,000 customers worldwide with aim to provide web transparency, and equal access to open-source, web-based information. Want to be featured on HackerNoon’s Company of the Week? Request Your Tech Company Page on HackerNoon! Bright Data’s Top Stories on HackerNoon Bright Data has published 9 stories via the HackerNoon Business Blogging Program, and has generated 20 days, 8 hours, and 29 minutes of reading time! Here are our top 3 favorite stories by them: How to Stay Anonymous Online by Hiding Your IP Address Alternatives to Web Scraping with Python Cracking the Code of Cloudflare Bypass Share Your Company's Story via HackerNoon That's all this week, folks! Stay Creative, Stay Iconic. The HackerNoon Team Another week, another Company of the Week ! Every week, we share an awesome tech brand from our tech company database , making their evergreen mark on the internet. This unique HackerNoon database ranks S&P 500 companies and top startups of the year alike. Company of the Week tech company database This week, we are proud to present Bright Data - one of the largest data collection platform on the planet internet having more than 15,000 customers worldwide with aim to provide web transparency, and equal access to open-source, web-based information. Bright Data Want to be featured on HackerNoon’s Company of the Week? Request Your Tech Company Page on HackerNoon ! Want to be featured on HackerNoon’s Company of the Week? Request Your Tech Company Page on HackerNoon Want to be featured on HackerNoon’s Company of the Week? Request Your Tech Company Page on HackerNoon Bright Data’s Top Stories on HackerNoon Bright Data has published 9 stories via the HackerNoon Business Blogging Program , and has generated 20 days, 8 hours, and 29 minutes of reading time! Here are our top 3 favorite stories by them: HackerNoon Business Blogging Program How to Stay Anonymous Online by Hiding Your IP Address How to Stay Anonymous Online by Hiding Your IP Address How to Stay Anonymous Online by Hiding Your IP Address Alternatives to Web Scraping with Python Alternatives to Web Scraping with Python Alternatives to Web Scraping with Python Cracking the Code of Cloudflare Bypass Cracking the Code of Cloudflare Bypass Cracking the Code of Cloudflare Bypass Share Your Company's Story via HackerNoon Share Your Company's Story via HackerNoon Share Your Company's Story via HackerNoon Share Your Company's Story via HackerNoon That's all this week, folks! Stay Creative, Stay Iconic. The HackerNoon Team

Meet Bright Data: HackerNoon Company of the Week

মিট ব্রাইট ডেটা: হ্যাকারনুন কোম্পানি অফ দ্য উইক

Lernen Sie Bright Data kennen: HackerNoon-Unternehmen der Woche

Conozca Bright Data: la empresa de la semana de HackerNoon

Rencontrez Bright Data : Entreprise HackerNoon de la semaine

ब्राइट डेटा से मिलिए: हैकरनून कंपनी ऑफ द वीक

Bright Data のご紹介: HackerNoon の今週の企業

밝은 데이터를 만나보세요: 금주의 HackerNoon 회사

Conheça Bright Data: Empresa HackerNoon da Semana

Встречайте Bright Data: компания недели HackerNoon

Bright Data ile Tanışın: Haftanın HackerNoon Şirketi

Gặp gỡ Bright Data: Công ty HackerNoon của tuần

结识 Bright Data：本周 HackerNoon 公司

REVIVING HOPES AND BRIGHTENING PROSPECTS.

Reflections on the Decline of Science in England, and on Some of Its Causes by Charles Babbage, is part of the HackerNoon Books Series. You can jump to any chapter in this book . Conclusion here CONCLUSION. In a work on the Decline of Science, at a period when England has so recently lost two of its brightest ornaments, I should hardly be excused if I omitted to devote a few words to the names of Wollaston and of Davy. Until the warm feelings of surviving kindred and admiring friends shall be cold as the grave from which remembrance vainly recalls their cherished forms, invested with all the life and energy of recent existence, the volumes of their biography must be sealed. Their contemporaries can expect only to read their eloge. In habits of intercourse with both those distinguished individuals, sufficiently frequent to mark the curiously different structure of their minds, I was yet not on such terms even with him I most esteemed, as to view his great qualities through that medium which is rarely penetrated by the eyes of long and very intimate friendship. Caution and precision were the predominant features of the character of Wollaston, and those who are disposed to reduce the number of principles, would perhaps justly trace the precision which adorned his philosophical, to the extreme caution which pervaded his moral character. It may indeed be questioned whether the latter quality will not in all persons of great abilities produce the former. Ambition constituted a far larger ingredient in the character of Davy, and with the daring hand of genius he grasped even the remotest conclusions to which a theory led him. He seemed to think invention a more common attribute than it really is, and hastened, as soon as he was in possession of a new fact or a new principle, to communicate it to the world, doubtful perhaps lest he might not be anticipated; but, confident in his own powers, he was content to give to others a chance of reaping some part of that harvest, the largest portion of which he knew must still fall to his own share. Dr. Wollaston, on the other hand, appreciated more truly the rarity of the inventive faculty; and, undeterred by the fear of being anticipated, when he had contrived a new instrument, or detected a new principle, he brought all the information that he could collect from others, or which arose from his own reflection, to bear upon it for years, before he delivered it to the world. The most singular characteristic of Wollaston's mind was the plain and distinct line which separated what he knew from what he did not know; and this again, arising from his precision, might be traced to caution. It would, however, have been visible to such an extent in few except himself, for there were very few so perfectly free from vanity and affectation. To this circumstance may be attributed a peculiarity of manner in the mode in which he communicated information to those who sought it from him, which was to many extremely disagreeable. He usually, by a few questions, ascertained precisely how much the inquirer knew upon the subject, or the exact point at which his ignorance commenced, a process not very agreeable to the vanity of mankind; taking up the subject at this point, he would then very clearly and shortly explain it. His acquaintance with mathematics was very limited. Many years since, when I was an unsuccessful candidate for a professorship of mathematics, I applied to Dr. W. for a recommendation; he declined it, on the ground of its not being his pursuit. I told him I asked it, because I thought it would have weight, to which he replied, that it ought to have none whatever. There is no doubt his view was the just one. Yet such is the state of ignorance which exists on these subjects, that I have several times heard him mentioned as one of the greatest mathematicians of the age. [This of course could only have happened in England.] But in this as in all other points, the precision with which he comprehended and retained all he had ever learned, especially of the elementary applications of mathematics to physics, was such, that he possessed greater command over those subjects than many of far more extensive knowledge. In associating with Wollaston, you perceived that the predominant principle was to avoid error; in the society of Davy, you saw that it was the desire to see and make known truth. Wollaston never could have been a poet; Davy might have been a great one. A question which I put, successively, to each of these distinguished philosophers, will show how very differently a subject may be viewed by minds even of the highest order. About the time Mr. Perkins was making his experiments on the compression of water, I was much struck with the mechanical means he had brought to bear on the subject, and was speculating on other applications of it, which I will presently mention. Meeting Dr. Wollaston one morning in the shop of a bookseller, I proposed this question: If two volumes of hydrogen and one of oxygen are mixed together in a vessel, and if by mechanical pressure they can be so condensed as to become of the same specific gravity as water, will the gases under these circumstances unite and form water? "What do you think they will do?" said Dr. W. I replied, that I should rather expect they would unite. "I see no reason to suppose it," said he. I then inquired whether he thought the experiment worth making. He answered, that he did not, for that he should think it would certainly not succeed. A few days after, I proposed the same question to Sir Humphry Davy. He at once said, "they will become water, of course;" and on my inquiring whether he thought the experiment worth making, he observed that it was a good experiment, but one which it was hardly necessary to make, as it must succeed. These were off-hand answers, which it might perhaps be hardly fair to have recorded, had they been of persons of less eminent talent: and it adds to the curiosity of the circumstance to mention, that I believe Dr. Wollaston's reason for supposing no union would take place, arose from the nature of the electrical relations of the two gases remaining unchanged, an objection which did not weigh with the philosopher whose discoveries had given birth to it. [The result of the experiment appeared, and still appears to me, to be of the highest importance; and I will shortly state the views with which it was connected. The next great discovery in chemistry to definite proportions, will be to find means of forming all the simple unions of one atom with one, with two, or with more of say other substance: and it occurred to me that the gaseous bodies presented the fairest chance of success; and that if wishing, for instance, to unite four atoms of one substance with one of another, we could, by mechanical means, reduce the mixed gases to the same specific gravity as the substance would possess which resulted from their union, then either that such union would actually take place, or the particles of the two substances would be most favourably situated for the action of caloric, electricity, or other causes, to produce the combination. It would indeed seem to follow, that if combination should take place under such circumstances, then the most probable proportion in which the atoms would unite, should be that which furnished a fluid of the least specific gravity: but until the experiments are made, it is by no means certain that other combinations might not be produced.] The singular minuteness of the particles of bodies submitted by Dr. Wollaston to chemical analysis, has excited the admiration of all those who have had the good fortune to witness his experiments; and the methods he employed deserve to be much more widely known. It appears to me that a great mistake exists on the subject. It has been adduced as one of those facts which prove the extraordinary acuteness of the bodily senses of the individual,—a circumstance which, if it were true, would add but little to his philosophical character; I am, however, inclined to view it in a far different light, and to see in it one of the natural results of the admirable precision of his knowledge. During the many opportunities I have enjoyed of seeing his minute experiments, I remember but one instance in which I noticed any remarkable difference in the acuteness of his bodily faculties, either of his hearing, his sight, or of his sense of smell, from those of other persons who possessed them in a good degree. [This was at Mr. South's observatory, and the object was, the dots on the declination circle of his equatorial; but, in this instance, Dr. Wollaston did not attempt to TEACH ME HOW TO SEE THEM.] He never showed me an almost microscopic wire, which was visible to his, and invisible to my own eye: even in the beautiful experiments he made relative to sounds inaudible to certain ears, he never produced a tone which was unheard by mine, although sensible to his ear; and I believe this will be found to have been the case by most of those whose minds had been much accustomed to experimental inquiries, and who possessed their faculties unimpaired by illness or by age. It was a much more valuable property on which the success of such inquiries depended. It arose from the perfect attention which he could command, and the minute precision with which he examined every object. A striking illustration of the fact that an object is frequently not seen, FROM NOT KNOWING HOW TO SEE IT, rather than from any defect in the organ of vision, occurred to me some years since, when on a visit at Slough. Conversing with Mr. Herschel on the dark lines seen in the solar spectrum by Fraunhofer, he inquired whether I had seen them; and on my replying in the negative, and expressing a great desire to see them, he mentioned the extreme difficulty he had had, even with Fraunhofer's description in his hand and the long time which it had cost him in detecting them. My friend then added, "I will prepare the apparatus, and put you in such a position that they shall be visible, and yet you shall look for them and not find them: after which, while you remain in the same position, I will instruct you how to see them, and you shall see them, and not merely wonder you did not see them before, but you shall find it impossible to look at the spectrum without seeing them." On looking as I was directed, notwithstanding the previous warning, I did not see them; and after some time I inquired how they might be seen, when the prediction of Mr. Herschel was completely fulfilled. It was this attention to minute phenomena which Dr. Wollaston applied with such powerful effect to chemistry. In the ordinary cases of precipitation the cloudiness is visible in a single drop as well as in a gallon of a solution; and in those cases where the cloudiness is so slight, as to require a mass of fluid to render it visible, previous evaporation, quickly performed on slips of window glass, rendered the solution more concentrated. The true value of this minute chemistry arises from its cheapness and the extreme rapidity with which it can be accomplished: it may, in hands like those of Wollaston, be used for discovery, but not for measure. I have thought it more necessary to place this subject on what I consider its true grounds, for two reasons. In the first place, I feel that injustice has been done to a distinguished philosopher in attributing to some of his bodily senses that excellence which I think is proved to have depended on the admirable training of his intellectual faculties. And, in the next place, if I have established the fact, whilst it affords us better means of judging of such observations as lay claim to an accuracy "MORE THAN HUMAN," it also opens, to the patient inquirer into truth, a path by which he may acquire powers that he would otherwise have thought were only the gift of nature to a favoured few. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Charles Babbage (1998).Reflections on the Decline of Science in England, and on Some of Its Causes. Urbana, Illinois: Project Gutenberg. Retrieved https://www.gutenberg.org/cache/epub/1216/pg1216-images.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at www.gutenberg.org https://www.gutenberg.org/policy/license.html

When England has so recently lost two of its brightest ornaments

Wives and Daughters by Elizabeth Cleghorn Gaskell, is part of the HackerNoon Books Series. You can jump to any chapter in this book . BRIGHTENING PROSPECTS. here CHAPTER XXXIII. BRIGHTENING PROSPECTS. It was a day or two afterwards, that Mr. Gibson made time to ride round by Hamley, desirous to learn more exact particulars of this scheme for Roger than he could obtain from any extraneous source, and rather puzzled to know whether he should interfere in the project or not. The state of the case was this:—Osborne's symptoms were, in Mr. Gibson's opinion, signs of his having a fatal disease. Dr. Nicholls had differed from him on this head, and Mr. Gibson knew that the old physician had had long experience, and was considered very skilful in the profession. Still he believed that he himself was right, and, if so, the complaint was one which might continue for years in the same state as at present, or might end the young man's life in an hour—a minute. Supposing that Mr. Gibson was right, would it be well for Roger to be away where no sudden calls for his presence could reach him—away for two years? Yet if the affair was concluded, the interference of a medical man might accelerate the very evil to be feared; and after all, Dr. Nicholls might be right, and the symptoms might proceed from some other cause. Might? Yes. Probably did? No. Mr. Gibson could not bring himself to say "yes" to this latter form of sentence. So he rode on, meditating; his reins slack, his head a little bent. It was one of those still and lovely autumn days when the red and yellow leaves are hanging-pegs to dewy, brilliant gossamer-webs; when the hedges are full of trailing brambles, loaded with ripe blackberries; when the air is full of the farewell whistles and pipes of birds, clear and short—not the long full-throated warbles of spring; when the whirr of the partridge's wings is heard in the stubble-fields, as the sharp hoof-blows fall on the paved lanes; when here and there a leaf floats and flutters down to the ground, although there is not a single breath of wind. The country surgeon felt the beauty of the seasons perhaps more than most men. He saw more of it by day, by night, in storm and sunshine, or in the still, soft, cloudy weather. He never spoke about what he felt on the subject; indeed, he did not put his feelings into words, even to himself. But if his mood ever approached to the sentimental, it was on such days as this. He rode into the stable-yard, gave his horse to a man, and went into the house by a side entrance. In the passage he met the Squire. "That's capital, Gibson! what good wind blew you here? You'll have some lunch? it's on the table, I only just this minute left the room." And he kept shaking Mr. Gibson's hand all the time till he had placed him, nothing loth, at the well-covered dining-table. "What's this I hear about Roger?" said Mr. Gibson, plunging at once into the subject. "Aha! so you've heard, have you? It's famous, isn't it? He's a boy to be proud of, is old Roger. Steady Roger; we used to think him slow, but it seems to me that slow and sure wins the race. But tell me; what have you heard? how much is known? Nay, you must have a glass full. It's old ale, such as we don't brew now-a-days; it's as old as Osborne. We brewed it that autumn, and we called it the young squire's ale. I thought to have tapped it on his marriage, but I don't know when that will come to pass, so we've tapped it now in Roger's honour." The old squire had evidently been enjoying the young squire's ale to the verge of prudence. It was indeed as he said, "as strong as brandy," and Mr. Gibson had to sip it very carefully as he ate his cold roast beef. "Well! and what have you heard? There's a deal to hear, and all good news, though I shall miss the lad, I know that." "I did not know it was settled; I only heard that it was in progress." "Well, it was only in progress, as you call it, till last Tuesday. He never let me know anything about it, though; he says he thought I might be fidgety with thinking of the pros and cons. So I never knew a word on't till I had a letter from my Lord Hollingford—where is it?" pulling out a great black leathern receptacle for all manner of papers. And putting on his spectacles, he read aloud their headings. "'Measurement of timber, new railings,' 'drench for cows, from Farmer Hayes,' 'Dobson's accounts,'—'um 'um—here it is. Now read that letter," handing it to Mr. Gibson. It was a manly, feeling, sensible letter, explaining to the old father in very simple language the services which were demanded by the terms of the will to which he and two or three others were trustees; the liberal allowance for expenses, the still more liberal reward for performance, which had tempted several men of considerable renown to offer themselves as candidates for the appointment. Lord Hollingford then went on to say that, having seen a good deal of Roger lately, since the publication of his article in reply to the French osteologist, he had had reason to think that in him the trustees would find united the various qualities required in a greater measure than in any of the applicants who had at that time presented themselves. Roger had deep interest in the subject; much acquired knowledge, and at the same time, great natural powers of comparison, and classification of facts; he had shown himself to be an observer of a fine and accurate kind; he was of the right age, in the very prime of health and strength, and unshackled by any family ties. Here Mr. Gibson paused for consideration. He hardly cared to ascertain by what steps the result had been arrived at—he already knew what that result was; but his mind was again arrested as his eye caught on the remuneration offered, which was indeed most liberal; and then he read with attention the high praise bestowed on the son in this letter to the father. The Squire had been watching Mr. Gibson—waiting till he came to this part—and he rubbed his hands together as he said,— "Ay! you've come to it at last. It's the best part of the whole, isn't it? God bless the boy! and from a Whig, mind you, which makes it the more handsome. And there's more to come still. I say, Gibson, I think my luck is turning at last," passing him on yet another letter to read. "That only came this morning; but I've acted on it already, I sent for the foreman of the drainage works at once, I did; and to-morrow, please God, they'll be at work again." Mr. Gibson read the second letter, from Roger. To a certain degree it was a modest repetition of what Lord Hollingford had said, with an explanation of how he had come to take so decided a step in life without consulting his father. He did not wish him to be in suspense for one reason. Another was that he felt, as no one else could feel for him, that by accepting this offer, he entered upon the kind of life for which he knew himself to be most fitted. And then he merged the whole into business. He said that he knew well the suffering his father had gone through when he had had to give up his drainage works for want of money; that he, Roger, had been enabled at once to raise money upon the remuneration he was to receive on the accomplishment of his two years' work; and that he had also insured his life, in order to provide for the repayment of the money he had raised, in case he did not live to return to England. He said that the sum he had borrowed on this security would at once be forwarded to his father. Mr. Gibson laid down the letter without speaking a word for some time; then he said,—"He'll have to pay a pretty sum for insuring his life beyond seas." "He's got his Fellowship money," said the Squire, a little depressed at Mr. Gibson's remark. "Yes; that's true. And he's a strong young fellow, as I know." "I wish I could tell his mother," said the Squire in an under-tone "It seems all settled now," said Mr. Gibson, more in reply to his own thoughts than to the Squire's remark. "Yes!" said the Squire; "and they're not going to let the grass grow under his feet. He's to be off as soon as he can get his scientific traps ready. I almost wish he wasn't to go. You don't seem quite to like it, doctor?" "Yes, I do," said Mr. Gibson in a more cheerful tone than before. "It can't be helped now without doing a mischief," thought he to himself. "Why, Squire, I think it a great honour to have such a son. I envy you, that's what I do. Here's a lad of three or four and twenty distinguishing himself in more ways than one, and as simple and affectionate at home as any fellow need to be—not a bit set up." "Ay, ay; he's twice as much a son to me as Osborne, who has been all his life set up on nothing at all, as one may say." "Come, Squire, I mustn't hear anything against Osborne; we may praise one, without hitting at the other. Osborne hasn't had the strong health which has enabled Roger to work as he has done. I met a man who knew his tutor at Trinity the other day, and of course we began cracking about Roger—it's not every day that one can reckon a senior wrangler amongst one's friends, and I'm nearly as proud of the lad as you are. This Mr. Mason told me the tutor said that only half of Roger's success was owing to his mental powers; the other half was owing to his perfect health, which enabled him to work harder and more continuously than most men without suffering. He said that in all his experience he had never known any one with an equal capacity for mental labour; and that he could come again with a fresh appetite to his studies after shorter intervals of rest than most. Now I, being a doctor, trace a good deal of his superiority to the material cause of a thoroughly good constitution, which Osborne hasn't got." "Osborne might have, if he got out o' doors more," said the Squire, moodily; "but except when he can loaf into Hollingford he doesn't care to go out at all. I hope," he continued, with a glance of sudden suspicion at Mr. Gibson, "he's not after one of your girls? I don't mean any offence, you know; but he'll have the estate, and it won't be free, and he must marry money. I don't think I could allow it in Roger; but Osborne's the eldest son, you know." Mr. Gibson reddened; he was offended for a moment. Then the partial truth of what the Squire said was presented to his mind, and he remembered their old friendship, so he spoke quietly, if shortly. "I don't believe there's anything of the kind going on. I'm not much at home, you know; but I've never heard or seen anything that should make me suppose that there is. When I do, I'll let you know." "Now, Gibson, don't go and be offended. I'm glad for the boys to have a pleasant house to go to, and I thank you and Mrs. Gibson for making it pleasant. Only keep off love; it can come to no good. That's all. I don't believe Osborne will ever earn a farthing to keep a wife during my life, and if I were to die to-morrow, she would have to bring some money to clear the estate. And if I do speak as I shouldn't have done formerly—a little sharp or so—why, it's because I've been worried by many a care no one knows anything of." "I'm not going to take offence," said Mr. Gibson, "but let us understand each other clearly. If you don't want your sons to come as much to my house as they do, tell them so yourself. I like the lads, and am glad to see them; but if they do come, you must take the consequences, whatever they are, and not blame me, or them either, for what may happen from the frequent intercourse between two young men and two young women; and what is more, though, as I said, I see nothing whatever of the kind you fear at present, and have promised to tell you of the first symptoms I do see, yet farther than that I won't go. If there's an attachment at any future time, I won't interfere." "I shouldn't so much mind if Roger fell in love with your Molly. He can fight for himself, you see, and she's an uncommon nice girl. My poor wife was so fond of her," answered the Squire. "It's Osborne and the estate I'm thinking of!" "Well, then, tell him not to come near us. I shall be sorry, but you will be safe." "I'll think about it; but he's difficult to manage. I've always to get my blood well up before I can speak my mind to him." Mr. Gibson was leaving the room, but at these words he turned and laid his hand on the Squire's arm. "Take my advice, Squire. As I said, there's no harm done as yet, as far as I know. Prevention is better than cure. Speak out, but speak gently to Osborne, and do it at once. I shall understand how it is if he doesn't show his face for some months in my house. If you speak gently to him, he'll take the advice as from a friend. If he can assure you there's no danger, of course he'll come just as usual, when he likes." It was all very fine giving the Squire this good advice; but as Osborne had already formed the very kind of marriage his father most deprecated, it did not act quite as well as Mr. Gibson had hoped. The Squire began the conversation with unusual self-control; but he grew irritated when Osborne denied his father's right to interfere in any marriage he might contemplate; denied it with a certain degree of doggedness and weariness of the subject that drove the Squire into one of his passions; and although, on after reflection, he remembered that he had his son's promise and solemn word not to think of either Cynthia or Molly for his wife, yet the father and son had passed through one of those altercations which help to estrange men for life. Each had said bitter things to the other; and, if the brotherly affection had not been so true between Osborne and Roger, they too might have become alienated, in consequence of the Squire's exaggerated and injudicious comparison of their characters and deeds. But as Roger in his boyhood had loved Osborne too well to be jealous of the praise and love which the eldest son, the beautiful brilliant lad, had received, to the disparagement of his own plain awkwardness and slowness, so now Osborne strove against any feeling of envy or jealousy with all his might; but his efforts were conscious, Roger's had been the simple consequence of affection, and the end to poor Osborne was that he became moody and depressed in mind and body; but both father and son concealed their feelings in Roger's presence. When he came home just before sailing, busy and happy, the Squire caught his infectious energy, and Osborne looked up and was cheerful. There was no time to be lost. He was bound to a hot climate, and must take all advantage possible of the winter months. He was to go first to Paris, to have interviews with some of the scientific men there. Some of his outfit, instruments, &c., were to follow him to Havre, from which port he was to embark, after transacting his business in Paris. The Squire learnt all his arrangements and plans, and even tried in after-dinner conversations to penetrate into the questions involved in the researches his son was about to make. But Roger's visit home could not be prolonged beyond two days. The last day he rode into Hollingford earlier than he needed to have done to catch the London coach, in order to bid the Gibsons good-by. He had been too actively busy for some time to have leisure to bestow much thought on Cynthia; but there was no need for fresh meditation on that subject. Her image as a prize to be worked for, to be served for seven years, and seven years more, was safe and sacred in his heart. It was very bad, this going away, and wishing her good-by for two long years; and he wondered much during his ride how far he should be justified in telling her mother, perhaps in telling her own sweet self, what his feelings were without expecting, nay, indeed reprobating, any answer on her part. Then she would know at any rate how dearly she was beloved by one who was absent; how in all difficulties or dangers the thought of her would be a polar star, high up in the heavens, and so on, and so on; for with all a lover's quickness of imagination and triteness of fancy, he called her a star, a flower, a nymph, a witch, an angel, or a mermaid, a nightingale, a siren, as one or another of her attributes rose up before him. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Elizabeth Cleghorn Gaskell (2003). Wives and Daughters. Urbana, Illinois: Project Gutenberg. Retrieved https://www.gutenberg.org/cache/epub/4274/pg4274-images.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at . www.gutenberg.org https://www.gutenberg.org/policy/license.html

BRIGHTENING PROSPECTS.

The stars still shone brightly when I awoke

In the heart of Saudi Arabia’s vast desert landscape, where sand dunes meet the shores of the Red Sea, lies a futuristic vision that aims to redefine the way we live - . This mass-scale mega-city project isn’t just about constructing a it’s about breathing life into a vision that combines cutting-edge technology with sustainable living to create an oasis like no other. NEOM smart city ; This grand endeavor seeks to establish a modern utopia, a place where imagination meets technology and the limits of possibility are pushed beyond what we thought was possible. So, let’s explore the potential this city holds to be the next big thing and whether it can spark a wave of similar smart city projects worldwide. The Rise of Smart Cities I believe that it’s an undeniable fact that smart cities are revolutionizing urban development and bringing a paradigm shift in the way we live and interact with one another. Smart cities are the manifestation of our collective aspiration to harmonize urban living with technological innovation. These cities to optimize resource allocation, enhance efficiency, and improve the quality of life for their citizens. leverage data-driven solutions Let’s look at some preceding NEOM to grasp what may be in store for us. smart city projects Songdo, South Korea Often known as the , Songdo is an international business district near Incheon. Designed from the ground up, Songdo is characterized by its cutting-edge technology integration and sustainable infrastructure. world’s first smart city With data being at the heart of all Seoul city projects, Songdo is no exception. There are monitoring everything from energy consumption to traffic flow and waste management in real-time, ensuring efficiency and environmental consciousness. The city also features a , through which waste is whisked away with the help of underground tubes. , sensors embedded throughout the city central pneumatic waste collection system Songdo's presence is a reminder that the ideals set forth by NEOM and other forward-thinking projects are already being realized elsewhere to enrich the lives of residents while setting new standards for urban development. Singapore Being named the “ ,” Singapore is one of the frontrunners in the global race for going ‘smart.’ With the city-state having long been recognized for its strategic urban planning and efforts to promote sustainability, Singapore’s transition to a smart city began with the launch of its “ ” initiative. This initiative aimed to leverage data and technology to improve citizens' lives. smartest city in Asia Smart Nation They introduced a number of smart technologies in both the private and public sectors. Ranging from a contactless payment system, to aid the millions of passengers availing public transport every day to the creation of robotic patrol police, and a facilitating online consultations and an IoT device to monitor patients’ health. , SimplyGo digital health system Singapore sets a precedent for the relationship between technology and urban living, standing out as a shining example of how embracing innovation can lead to sustainable and harmonious urban environments. NEOM: A Glimpse into the Future NEOM isn't just a city; it’s aimed to be a manifesto for urban development. The vision is to create an oasis where humans and technology can coexist synergistically. Let’s explore what plans the Saudi government has for this colossal smart city project. About NEOM NEOM is a smart city project planned by the Kingdom of Saudi Arabia as part of its initiative. This plan aims to reduce Saudi Arabia’s dependence on oil, diversify its economy, and boost the country’s public and private sectors. Saudi Vision 2030 NEOM is to be built in the country’s northwestern Tabuk province, north of the Red Sea and in close proximity to the east of Egypt and the south of Jordan. The area for the project is 26,500km2 and currently encompasses four different planned regions so far: Sindalah Nestled along the Red Sea coast, is an island destination that embodies the spirit of leisure, luxury, and recreation. With pristine beaches, luxury resorts, and a thriving entertainment scene, this region is envisioned as a haven for relaxation and entertainment. It offers a plethora of underwater activities and will also be a yachting hub for some of the world’s most prominent yachting enthusiasts with its 86-berth marina and 75 off-shore buoys. Sindalah Trojena emerges as a dynamic ski resort with elevations ranging from 1,500 to 2,600 meters and a pristine expanse spanning nearly 60 square kilometers. Trojena offers an enchanting playground for winter sports enthusiasts. Blanketed in snow during sub-zero winters, this alpine haven promises exhilarating slopes, breathtaking panoramas, and an unforgettable skiing experience that embodies the spirit of adventure within NEOM's visionary landscape. Trojena Oxagon Previously known as Neom Industrial City, is an octagonal floating industrial complex. Situated around 25 kilometers north of Duba, Oxagon envelops a sprawling expanse of 200–250 square kilometers. This marvel is set to redefine modern manufacturing and industrial research, positioning itself as a dynamic hub for progress centered around the expansion of Duba port. Expected to be powered by 100% renewable energy, Oxagon's blueprint includes a desalination plant, a hydrogen facility, and an oceanographic research center, emphasizing NEOM's commitment to sustainable practices and scientific exploration. Oxagon The Line At the heart of NEOM's vision lies , a linear city stretching across 170 kilometers. The Line promises to be a blueprint for sustainability and efficiency. Free from the constraints of conventional urban design, it will be powered by 100% renewable energy and will feature cutting-edge transportation systems and AI-powered services. The Line's unique structure emphasizes proximity to nature, ensuring that no resident is more than a five-minute walk away from green spaces, offering a harmonious blend of urban and natural environments. The Line NEOM’s Impact Right now, the whole world is keeping a close eye on NEOM and waiting to see how it unfolds. If the project turns out to be a success, it could have implications spanning far and wide. Economic and Societal Impact The economic and societal impact of NEOM will be unprecedented. Job creation in emerging sectors such as data analytics, cybersecurity, and could reshape the employment landscape. NEOM is expected to create hundreds of thousands of job opportunities for locals and immigrants alike in the country. The Line alone is by 2030. AI programming expected to create 380,000 jobs Furthermore, the very essence of smart cities, which are convenience, efficiency, and sustainability, has the potential to elevate citizens' well-being. The findings of a by the McKinsey Global Institute show that cities that use smart technologies improve some key quality-of-life indicators by 10 to 30 percent. This contributes to shorter commutes, fewer crimes, a reduced health burden, and more lives saved. In addition to that, when citizens are freed from mundane tasks by automation and empowered by technology, they can focus on creative pursuits, personal growth, and contributing meaningfully to society. report Global Implications NEOM's success could also potentially send ripples across the globe, inspiring other nations to follow suit. As we witness various countries embarking on their smart city journeys, NEOM's innovative approach might serve as a benchmark for what's possible. From China's tech-driven city of to India's plans for sustainable urban development in , the world is closely watching and learning from NEOM. Xiong'an Lavasa Challenges facing NEOM The path to smart city utopia isn’t without thorns, and NEOM faces a spectrum of challenges and severe criticism that demand careful consideration. NEOM has not been without its share of controversies, with notable issues including the from their ancestral lands and allegations of human rights violations against migrant laborers. forced displacement of the Huwaitat tribe Critics have also voiced legitimate concerns about the feasibility and long-term sustainability of NEOM. The sheer magnitude of the project, coupled with its potential environmental repercussions, has prompted valid inquiries. Realizing the city’s grand ambitions, such as achieving full reliance on , building a floating trade hub, and implementing autonomous transportation, presents substantial challenges. renewable energy The geopolitical landscape surrounding NEOM introduces an additional layer of uncertainty. Given its proximity to Egypt and Jordan, the intricate dynamics of the region add complexity that could influence the project's outcomes. Then, there are also concerns about the amalgamation of technology in the design for NEOM. A prominent reason why Songdo didn’t achieve the success it was meant for was its . The city was designed with technology in mind, keeping humans in the peripheral rather than keeping the needs of humans in mind and using technology to fulfill those needs. NEOM must avoid making the same mistake. technology-centric design Will Smart Cities Be the Next Big Thing? As the sun sets over the desert, NEOM illuminates the horizon with countless possibilities. In the wake of this transformative vision, the question looms: Will smart cities become the next big thing? As we witness the convergence of , sustainability, and urban living, it's undeniable that smart cities hold immense promise. NEOM's grand endeavor underscores the potential for harmonizing innovation and nature. However, for this vision to thrive globally, challenges like environmental impact, inclusivity, and geopolitical complexities must be navigated with wisdom. technology The path forward demands a delicate balance between progress and responsibility, ensuring that the evolution of urban landscapes becomes a testament to human advancement that respects our planet and fosters shared prosperity.

Saudi NEOM Sparks a Brighter Future: Will Smart Cities Be the Next Big Thing?

সৌদি NEOM একটি উজ্জ্বল ভবিষ্যত তৈরি করে: স্মার্ট শহরগুলি কি পরবর্তী বড় জিনিস হবে?

Saudi-Arabiens NEOM eröffnet eine bessere Zukunft: Werden Smart Cities das nächste große Ding sein?

NEOM saudita genera un futuro más brillante: ¿Serán las ciudades inteligentes la próxima gran novedad?

NEOM saoudien ouvre un avenir meilleur : les villes intelligentes seront-elles la prochaine grande nouveauté ?

सऊदी NEOM एक उज्जवल भविष्य की आशा करता है: क्या स्मार्ट शहर अगली बड़ी चीज़ होंगे?

サウジのNEOMが明るい未来を拓く: スマートシティが次の目玉となるのか?

사우디 NEOM은 더 밝은 미래를 촉발합니다: 스마트 시티가 차세대 혁신이 될까요?

NEOM saudita desperta um futuro mais brilhante: as cidades inteligentes serão a próxima grande novidade?

Suudi NEOM Daha Parlak Bir Geleceğe Yol Açıyor: Akıllı Şehirler Bir Sonraki Büyük Şey mi Olacak?

Saudi NEOM khơi dậy một tương lai tươi sáng hơn: Liệu các thành phố thông minh có trở thành điều lớn lao tiếp theo không?

沙特 NEOM 开启更光明的未来：智慧城市会成为下一个重大事件吗？

Nowadays, many applications help us to edit our photos and images. With these applications, we can add some artistic effect or a filter that makes our image look with better color or maybe without color or look like an old photo, etc. Apps such as Instagram offer editing options with many predefined filters and the option to create one by ourselves. This makes me wonder: “How do filters work on digital images? How are they created, or what do they do to the image?”. I’m not going to address issues such as image processing with graphics or algorithms, but I would like to develop an application that allows us to add filters to an image. Using web technologies and a cool repository, I found we will develop a clone of the Instagram filters. What is a filter on an image? A filter is a mathematical equation (called a convolution product) that allows modifying the value of a pixel according to the values of contiguous pixels, with coefficients for each pixel in the region to which it is applied. Different objectives: Smooth the image: reduce intensity variations between neighboring pixels. Eliminate noise: modify those pixels whose intensity level is very different from that of their neighbors. Enhance the image: increase intensity variations where they occur. Detect edges: detect those pixels where there is an abrupt change in the intensity function. The CSS Filter property provides graphical effects such as blurring or color change in the rendering before the element is displayed. Filters are commonly used to adjust the rendering of images, backgrounds, or borders. The CSS filter property The functions that we will use of this property to create our filters are: This is the difference in light intensity between the black and white tones of the image. High contrasts darken shadows and dark tones more, making whites brighter, while low contrasts decrease the intensity of both ranges. : contrast() : The attribute of a visual sensation whereby an area appears to show more or less light. brightness() : It is the intensity of each color in a photo according to its composition’s degree of white, black, or gray. The more absence of these on a color, the more purity and intensity. And the more it is mixed with gray, the less saturation. saturate() : A sepia image is a black-and-white image that has a warm brown tone. Sepia tints can make an image look older. sepia() : Grayscale is a range of monochrome shades of black to white. Therefore, a grayscale image contains only shades of gray and no color. grayscale() Time to Code As I mentioned before, we will use , which is like a CSS class library that provides us with filters very similar to Instagram’s. You can find the repository and demo of the application we will develop at the end of the tutorial. Instagram.css Setting Up We create our application with the following command: npx create-react-app myApp We install the dependencies that we will need in the project: npm i react-slick slick-carousel rc-slider styled-components react-icons to create a Slider with the different CSS filters. react-slick and react-carousel to create the range sliders for our custom styles. rc-slider to style our app. styled-components icons for some titles and buttons. react-icons After that, we create the following structure for the project: src/
├── assets/
│   ├── fonts
│   └── img
├── components/
│   ├── CustomFilter.jsx
│   ├── CustomFilterOptions.jsx
│   ├── Filter.jsx
│   ├── ImgWrapper.jsx
│   ├── NormalFilter.jsx
│   └── Title.jsx
├── pages/
│   └── Home.js
├── styles/
│   └── GlobalStyles.js
│   ├── instagram.min.css
│   └── Typography.js
├── App.js
└── index.js As you can see, it will be a simple application where what we want to review is how to add the CSS classes to our previously added image in the folder. assets/img Also, develop functionalities with which we will be able to create a customized filter. Components : It is the root component, where we will create a couple of states which will allow us to know which filter is selected at the moment and thus apply it to our main image. Home.jsx It also has a state that will be responsible for saving the custom filters that the user will select through the range of Sliders that we are going to create. : In this component, we import the file and we map the names of the classes that the library offers us. I have created a JSON object which contains all the available classes. Filters.jsx instagram.min.css const filters = [
  { name: 'Aden', class: 'filter-aden'},
  { name: 'Amaro', class: 'filter-amaro'},
  { name: 'Ashby', class: 'filter-ashby'},
  { name: 'Brannan', class: 'filter-brannan'}
  ...
] Now with the filters in an object, we will make a map and create for each element a card with the image we want to edit, adding the corresponding class to give a preview of the added effect. In addition to that with the library, we have created a Slider so we can visualize all the elements. react-slick import React, { useEffect } from 'react'
import styled from 'styled-components'
import Slider from 'react-slick'

import thumbnail from '../assets/img/img1.jpg'

import '../styles/instagram.min.css'
import 'slick-carousel/slick/slick.css'
import 'slick-carousel/slick/slick-theme.css'

const FiltersStyles = styled.div`
  width: 70%;
  margin: auto;
  .slick-prev:before {
    color: var(--blue-1);
  }
  .slick-next:before {
    color: var(--blue-1);
  }
`
const Filters = ({ filterClass, setFilterClass, imgRef }) => {
  useEffect(() => {
    const divImg = imgRef.current
    divImg.style.filter = ''
  }, [filterClass])

  const filters = [
    {
      name: 'Aden',
      class: 'filter-aden',
    },
    {
      name: 'Amaro',
      class: 'filter-amaro',
    },
    {
      name: 'Ashby',
      class: 'filter-ashby',
    },
  ]

  const settings = {
    dots: false,
    Infinite: true,
    speed: 500,
    slidesToShow: 5,
    slidesToScroll: 5,
  }

  return (
    <FiltersStyles>
      <Slider {...settings}>
        {filters.map((filter, index) => {
          return (
            <div key={index}>
              <div
                className={`filter-item ${
                  filterClass === filter.class ? 'filter-item--selected' : ''
                }`}
                onClick={() => setFilterClass(filter.class)}
              >
                <div className="filter-item__img">
                  <img
                    className={filter.class}
                    src={thumbnail}
                    alt={filter.name}
                  />
                </div>
                <div className="filter-item__name">
                  <p>
                    <strong>{filter.name}</strong>
                  </p>
                </div>
              </div>
            </div>
          )
        })}
      </Slider>
    </FiltersStyles>
  )
}

export default Filters : In this component, we try to create controls that allow us to create our effect. For this case, we use the CSS Filter property, which provides us with several interesting functions. CustomFilterOptions.jsx So that our app is more similar to the controls of the already existing applications, I have used the library with which we will create ranges of values with which each function of the Filter property works. rc-slider We will create a state and initialize it with the starting value of each effect. Using the Sliders, we will save each newly selected value with the property of each component. onChange Finally, we have a function that we call with which we will create our CSS style starting from the values of each effect that we create and if we assign it to our main image. setCustomFilterClass With this, the effect will be created in our image. import React, { useState } from 'react'
import Slider, { SliderTooltip } from 'rc-slider'
import styled from 'styled-components'

import 'rc-slider/assets/index.css'

const CustomFilterOptionsStyles = styled.div`
  .customFilters-container {
    width: 400px;
    margin-right: 100px;
  }
  .customFilters-item {
    margin: 3rem 0;
    display: flex;
    gap: 1rem;
    align-content: center;
    p {
      width: 160px;
    }
  }
`

const CustomFilterOptions = ({ imgRef }) => {
  const [contrast, setContrast] = useState(100)
  const [brightness, setBrightness] = useState(100)

  const { Handle } = Slider

  const handle = props => {
    const { value, dragging, index, ...restProps } = props

    return (
      <SliderTooltip
        prefixCls="rc-slider-tooltip"
        overlay={`${value} %`}
        visible={true}
        placement="top"
        key={index}
      >
        <Handle value={value} {...restProps} />
      </SliderTooltip>
    )
  }

  const onChangeContrast = value => {
    setContrast(value)
    setCustomFilterClass()
  }

  const onChangeBrightness = value => {
    setBrightness(value)
    setCustomFilterClass()
  }

  const setCustomFilterClass = () => {
    const style = `contrast(${contrast}%) brightness(${brightness}%) saturate(${saturate}%) sepia(${sepia}%) grayScale(${gray}%)`

    const divImg = imgRef.current
    divImg.style.filter = style
  }

  return (
    <CustomFilterOptionsStyles>
      <div className="customFilters-container">
        <div className="customFilters-item">
          <p>Contrast</p>
          <Slider
            step={1}
            min={0}
            max={200}
            defaultValue={contrast}
            handle={handle}
            onChange={onChangeContrast}
          />
        </div>
        <div className="customFilters-item">
          <p>Brightness</p>
          <Slider
            step={1}
            min={0}
            max={200}
            defaultValue={brightness}
            handle={handle}
            onChange={onChangeBrightness}
          />
        </div>
      </div>
    </CustomFilterOptionsStyles>
  )
}

export default CustomFilterOptions That’s it! We have created an application with which we can assign Instagram-like effects to any image we want, in addition to being able to create our custom effect. See the demo here Project repo here Conclusion In this tutorial, we tried to simulate the operation of applications that edit images by adding filters or effects to them, in this case, Instagram-like filters. Although this application can still be improved and add many other basic features that these types of apps have, we have developed the part that interested us about the filters. I hope you had fun developing this app as I did and that you can add these features to any project that comes your way. Read more: How to Build a Shopping Cart with Next.js and Zustand: State Management with TypeScript Protect Your Application by Hiding API Keys and Tokens in React Want to connect with the Author? Love connecting with friends all around the world on . Twitter Also published . here

Enhance Your Photos with the CSS Contrast, Brightness, Saturation, and Sepia Functions

You are reading the third part of a three-part series. It is highly encouraged to that you read the previous two articles before proceeding with this part! Flix-Finder [Part 1/3] Flix-Finder [Part 2/3] By the end of the previous blog, Flix-Finder supported movie searches based on rating parameters from various agencies. The functionality of our app met expectations and achieved its intended function. But after a while, Flix-Finder's customers began to gripe that the app did not provide the most recent information about films. Because Flix-Finder employs static data, this is something we anticipate. Flix-Finder needs some process through which its movie data can be updated to reflect the new movies. And, this is something we would be implementing in this final part! Despite the fact that there are several ways to implement this functionality, we'll be creating a new endpoint that can be used to update the data. Your data would change instantaneously, much as when you press a button. But there are other strategies as well, such utilising a cronjob. Once the specifics of one strategy are described in this blog, any new technique may be easily created using the same concepts. Planning it all out… Before we start to code, let us take a step back and understand all our requirements. We want to create a new endpoint and hitting that endpoint should update the data Flix-Finder uses. Let us use endpoint and any hitting this endpoint should update the data our app uses. /update GET request With the interface planned out, let us now think about ‘ . ‘ How to update the data? The process of updating the data can be broken into three steps: (1) Re-Run the Collectors on Bright Data (2) Fetch the Results obtained from running the collector (3) Overwrite the results to static files For the first two steps, we can use the APIs provided by Bright Data. Once we have the data, we can use vanilla Python to overwrite the existing static files. Great! With everything planned out, let us start the implementation. Adding the new Endpoint to Flix-Finder Start by creating a new method inside the file of recommender app in . We will call this method . This is where all our logic of updating the static data will reside. view.py Django updateData # recommender/views.py

def update(request):
  # TODO: Implement this method
  return HttpResponse("Movie Data Updated!") Let us set up our views as well. We need to add the mapping for the newly created method in views. # recommender/urls.py

from django.urls import path

from . import views

urlpatterns = [
    path("", views.index, name = "index"),
    path("update", views.updateData, name = "update data")
] With this, we have successfully created a new endpoint in Flix-Finder. We will be using this endpoint to update the static data. For now, this endpoint does not do anything and simply returns a String message. ➜  ~ curl localhost:8000/update        
Movie Data Updated! Generating API Token on Bright Data Before we can start using Bright Data’s API for data update, we need to generate an API Token. All communications to Bright Data should contain this token and it is used for authentication purposes. Steps to generate API Token : Head over to . https://brightdata.com/cp/setting In the API Token section, click on button. Add Token In the dialog box, select the user, the permission and the expiration date. Click on the save button and enter the 2FA code. Copy and securely keep the generated token. Make sure to not leak your API Token. Anyone with access to your API Token can literally mimic you on Bright Data!!! Once the API Token is generated, we are ready to start using the APIs for running the collectors and fetching data from collector runs. APIs for Initiating Collectors Head over to Initiate by API section of your collectors. Here you will find curl requests that you can use to re-run collectors and fetch the results. Bright Data provides an endpoint to initiate collector run. The endpoint expects a POST request. It also expects a query parameter ‘ ’ that is used to uniquely identify the collector. The header should contain the API token. API to initiate collector run: https://api.brightdata.com/dca/trigger_immediate collector Here is the curl request for the same. curl -H "Authorization: Bearer API_TOKEN" -H "Content-Type: application/json" -d '{}' "https://api.brightdata.com/dca/trigger_immediate?collector=c_rj3i0wspnhr0ataba" The response of this endpoint is a response ID. This can be used to later fetch the results. With the response ID obtained from running a collector, Bright Data provides us with another API to fetch results from the collector run. The endpoint expects a GET request. It also expects a query parameter ‘response_id’ that is used to uniquely identify the collector. The header should contain the API token. API to fetch results from a collector’s run: https://api.brightdata.com/dca/get_result Here is how the curl request for this endpoint would look. curl "https://api.brightdata.com/dca/get_result?response_id=RESPONSE_ID" -H "Authorization: Bearer API_TOKEN" Implementing the update endpoint With the knowledge of endpoints to use and API token handy, we can start implementing the update endpoint. Let us implement this endpoint according to the subtasks we identified initially. (1) Re-Run the Collectors on Bright Data (2) Fetch the Results obtained from running the collector (3) Overwrite the results to static files First we need to use the re-run collector API and get a response ID. We will be using Python’s requests library to perform this task. API_TOKEN = "###########" # Replace by your API Token
response = requests.post(
    "https://api.brightdata.com/dca/trigger_immediate?collector=c_rj3i0wspnhr0ataba",
    headers={"Authorization": f"Bearer {API_TOKEN}", "Content-Type": "application/json"},
    json={}
)
if response.status_code < 200 or response.status_code > 299 or 'response_id' not in response.json():
    return HttpResponseServerError('Something Wrong Happened')
response_id = response.json().get('response_id') Next step is to use the response ID to fetch the results. We would wait for some time before trying to fetch the results. time.sleep(30) # Sleeping for 30 seconds
get_results_response = requests.get(
    f"https://api.brightdata.com/dca/get_result?response_id={response_id}",
    headers={"Authorization": f"Bearer {API_TOKEN}"}
) With data in hand, we can overwrite the static file with this new data. movie_data = get_results_response.json()
data_file = open(ROTTEN_TOMATO_DATA_FILE_PATH, encoding="utf8", mode="w")
json.dump(movie_data, data_file, indent=4)
data_file.close() With that the implementation of the update method is completed. Using the new endpoint A simple curl request can be made to the endpoint to initiate the data updation process. Once the API returns, the data would have been updated. localhost:8000/update ➜  ~ curl localhost:8000/update        
Movie Data Updated! Concluding Remarks Finally, we have reached the conclusion of this section. In the Flix-Finder, we were able to integrate data updating functionality. As a result, our Flix-Finder will always be available to its users with fresh data! I hope you liked the three-part series. This contains the whole collection's source code. Github repository You can for more such amazing articles. See you in the next article… Till then, Happy Learning! 🙂 follow me on HackerNoon

Flix-Finder: Building a Movie Recommendation App With Django and Bright Data [Part 3/3]

ফ্লিক্স-ফাইন্ডার: জ্যাঙ্গো এবং উজ্জ্বল ডেটা সহ একটি মুভি সুপারিশ অ্যাপ তৈরি করা [পর্ব 3/3]

Flix-Finder: Erstellen einer Filmempfehlungs-App mit Django und Bright Data [Teil 3/3]

Flix-Finder: Creación de una aplicación de recomendación de películas con Django y Bright Data [Parte 3/3]

Flix-Finder : créer une application de recommandation de films avec Django et Bright Data [Partie 3/3]

फ़्लिक्स-फ़ाइंडर: Django और ब्राइट डेटा के साथ एक मूवी अनुशंसा ऐप बनाना [भाग 3/3]

Flix-Finder: Django と Bright Data を使用した映画おすすめアプリの構築 [パート 3/3]

Flix-Finder: Django와 Bright Data로 영화 추천 앱 만들기 [3/3부]

Flix-Finder: Construindo um aplicativo de recomendação de filme com Django e Bright Data [Parte 3/3]

Flix-Finder: Django ve Parlak Verilerle Film Önerisi Uygulaması Oluşturma [Bölüm 3/3]

Flix-Finder: Xây dựng ứng dụng đề xuất phim với dữ liệu Django và Bright [Phần 3/3]

Flix-Finder：使用 Django 和 Bright Data 构建电影推荐应用程序 [第 3/3 部分]

You are reading the second part of a three-part series. It is highly encouraged to read the first article before proceeding! here We started working on “ ” with a vision of developing a software solution to the problem of picking a movie based on numerous rating parameters. By the end of the previous part of this series, enabled us to filter films based on release year and minimum IMDB rating. But as we argued, our application will stand true to its existence only if it can filter movies across several rating parameters. Flix-Finder Flix-Finder Today, in the second part of this series, we will be adding support for filtering movies based on Rotten Tomato ratings as well! With this feature implemented, our application would be able to search movies based on two different rating parameters. Additionally, with the method outlined in this article, you (the reader) can add as many other rating parameters as you want! Let us get coding… Rotten Tomato Rating Data Our software utilizes information gathered from multiple rating agencies. For the first section, we used pre-curated IMDB rating information from . The same thing is required for Rotten Tomatoes. Bright Data A quick search indicates that Rotten Tomato does not offer a developer API for retrieving such information. Additionally, Bright Data does not offer a pre-curated dataset for Rotten Tomatoes. No! We have a route out in the form of Bright Data’s custom web scrapers! Does this imply that we are at a dead end? Bright Data’s Custom Collectors Bright Data’s collector is a tool that can be used to collect web data at large scales. It provides an integrated development environment (IDE) that enables the automation of any data-collecting procedure. This environment comes with a sizable number of pre-defined functions to make some of the typical data-collecting tasks easier. What, then, is a Collector? Before writing our own custom collector for collecting Rotten Tomatoes rating data, we should first understand some terminologies surrounding these collectors. A "stage" is a particular phase of data collecting. When users wish to gather data from several places or want to carry out numerous activities while collecting data, stages can be added. Users may more easily control and monitor the development of their data-gathering efforts by segmenting the total data collection process into several stages. For instance, a YouTube data collector may contain steps like searching, playing each video that comes up in the search results, and retrieving all of the comments. Collector Stages: In Bright Data IDE, the "interaction code" refers to the code that is executed during the data collection process to interact with the web page. It is a code written in JavaScript that runs within the context of the web page being scraped. The interaction code can be used to perform a wide range of actions, such as clicking buttons, scrolling through the page, and filling out forms. Interaction Code: It contains the logic for parsing the HTML from the interactions made in the interaction code. We use the method from the interaction code to call the parser code. Parser Code: parse() And that is all we need to know to get started with writing our first collector! Collector For Rotten Tomato Having understood the theory, let us now move on to the practical. We will be using data from Rotten Tomato's . This list contains a large number of movie ratings (~2000) across various genres. all-time lists So this is how we will structure stages for our collector: The first stage would be responsible for opening various lists from the all-time lists page. The second stage would be responsible for collecting movie names, ratings, and years of release. Stage 1: Opening Movie List According to the description above, this stage should be able to navigate to the all-time lists page, capture the URLs for all the lists on the page, and pass them one by one on to the next stage. This is how the interaction code for this stage will look like. let url = new URL('https://editorial.rottentomatoes.com/all-time-lists/') // (1)
navigate(url) // (2) 
let hrefArray1 = parse().hrefArray; // (3) 
hrefArray1.forEach(href => next_stage({  // (4)
  url: href
}))

let url2 = new URL('https://editorial.rottentomatoes.com/all-time-lists/?wpv_view_count=1773-CATTR9a61a7f98d435e1c32de073e05574776&wpv_paged=2');
navigate(url2);
let hrefArray2 = parse().hrefArray;
hrefArray2.forEach(href => next_stage({
  url: href
})) Let us understand this code, line-by-line. We create a new URL object. This class is provided by default in the IDE by Bright Data. We provide the URL for the all-time list as a parameter. We navigate to this URL using another function provided. This is essentially equivalent to opening a web page. We next call the parser code that will parse the data and return a list of URLs. These URLs correspond to different lists on the page. We then call the next stage for each of the lists. Yet again, IDE provides us with the next_stage method to send inputs to the next stage. We repeat these same steps for another URL that corresponds to the second page of the all-time list. Next, let us look at the parser code for this stage. var hrefArray = []; //create an empty array to store href values

$("div.col-sm-8.newsItem.col-full-xs a.unstyled.articleLink").each(function() {

  var href = $(this).attr("href"); //get the href attribute value of current 'a' tag

  hrefArray.push(href); //push href value to the array

});



return {hrefArray}; The parser logic is pretty simple. We have written a selector for selecting all the URLs from the different lists available on the page. Stage 2: Collecting Movie Data The second stage receives a URL as input. This URL refers to the web page containing a list of movies. The responsibility of this stage should be to navigate to the provided URL and collect all the movie data available on that page. Let us look at the interaction code for it. navigate(input.url);
let {movieData} = parse();
movieData.forEach(movie => collect(movie)) As described, we navigate to the provided URL and call the parser. The parser returns a list of movie data. We call the method to “collect“ that data! collect This is what the parser code looks like. let movieData = []
$('div[id^="row-index-"]').each(function() {
  let posterSrc = $(this).find("div > a.article_movie_poster > div > img").attr("src");
  let movieName = $(this).find('div.article_movie_title div > h2 > a').text();
  let movieYear = $(this).find('span.subtle.start-year').text().slice(1, -1);
  let movieRating = $(this).find('span.tMeterScore').text();
  movieData.push({posterSrc, movieName, movieYear, movieRating})
});

return {movieData} A quick look at the parser code reveals that we are selecting four properties for each of the movies on the page. We then create a list of all such movies and return that from the parser. Great! We just finished writing our custom collector! To test that everything is working, click on the preview button (as highlighted in the image below). Once the preview is successful, you can click on the Save button on the top-right. Our collector is now saved and ready to use. Running our Collector Select the MyScrapers tab from the Datasets & WebScraper IDE menu. Your recently created collector will be visible here. Click on the three dots towards the left of your collector and select initiate manually. Click ‘Start’ on the ‘Initiate manually’ menu. This will start the collector. Wait for the process to finish. Once the run is finished, we can download the JSON file under the Actions. Check that this file contains roughly 2000 movies! Rename the downloaded file to and place it under the static folder of Django app. This folder should already contain the IMDB data file and the CSS file for our application. rtm.json recommender With all this, the data for our application is ready, and we can begin updating our application! Adding an extra field to our Django Form Continuing from the form we created in the last part, we will be adding three new fields to our form. Following will be the fields - (1) : Sets a threshold for the minimum Rotten Tomato Rating Minimum Rotten Tomato Rating (2) : Checkbox to indicate if we want to include those movies for which Rotten Tomato ratings exist and meet the threshold, but IMDB ratings are not present. Include Empty IMDB Ratings (3) : Checkbox to indicate if we want to include those movies for which IMDB ratings exist and meet the threshold, but Rotten Tomato ratings are not present. Include Empty Rotten Tomato Ratings Here is what the updated class looks like. from django import forms

class MovieSearchForm(forms.Form):
    imdb_rating = forms.DecimalField(min_value=0.0, max_value=9.9, label='Minimum IMDB Rating')
    rotten_tomato_rating = forms.DecimalField(min_value=0.0, max_value=9.9, label='Minimum Rotten Tomato Rating')
    include_empty_rotten_tomato_ratings = forms.BooleanField()
    include_empty_imdb_ratings = forms.BooleanField()
    release_year = forms.IntegerField(required=True) With this change, you should be able to see the updated form on the webpage! Updating the View With both the data and the form in place, what remains is to update our view so that recommendations can be made considering the Rotten Tomato ratings as well. Let us first write a method that can read the static JSON file and create a list of Python objects representing Rotten Tomato Rating data. # recommender/views.py

import json

ROTTEN_TOMATO_DATA_FILE_PATH = './recommender/static/rtm.json'

class RottenTomatoMovieData:
    def __init__(self, movie_title, rating, release_year) -> None:
        self.movie_title = movie_title
        try: 
            self.rating = float(rating.strip('%')) / 10
        except ValueError:
            self.rating = 0
        try:
            self.release_year = int(release_year)
        except ValueError:
            self.release_year = None

def _getRottenTomatoMovieDataList():
    rotten_tomato_movie_list = []
    data_file = open(ROTTEN_TOMATO_DATA_FILE_PATH, encoding="utf8")
    json_data = json.load(data_file)
    for movie_json_data in json_data:
        if 'movieName' not in movie_json_data:
            continue
        rotten_tomato_movie_data = RottenTomatoMovieData(movie_json_data['movieName'], movie_json_data['movieRating'], movie_json_data['movieYear'])
        rotten_tomato_movie_list.append(rotten_tomato_movie_data)
    data_file.close()
    return rotten_tomato_movie_list We created a new class to represent the rating data obtained from Rotten Tomato. The parsing code is pretty simple, we read the JSON file and convert each JSON object one-by-one to . RottenTomatoMovieData RottenTomatoMovieData The next thing we need to do is filter this list based on the values that the user entered in the form. Let's create another function that will accept the form data and the list of Rotten Tomatoes movie ratings. The filtered list would then be returned by this function. # recommender/views.py

def _filter_rotten_tomato_movie_list(rotten_tomato_movie_list, movie_search_form: MovieSearchForm):
    filtered_list = []
    for movie in rotten_tomato_movie_list:
        if movie_search_form.cleaned_data['rotten_tomato_rating'] and movie.rating < movie_search_form.cleaned_data['rotten_tomato_rating']:
            continue
        if movie.release_year and movie.release_year != movie_search_form.cleaned_data['release_year']:
            continue
        filtered_list.append(movie)
    return filtered_list Again, the filtering logic is straightforward. We iterate over the list of movies and filter out any movie whose rating does not match the threshold or the release year is different from what is requested. Merging Movie Data We have a list of movies from IMDB rating data after part 1. This list was used to present the results on our website. Using the Rotten Tomatoes ratings, we have now produced another list of movies. These two lists must be combined in some way to produce a single list that can be used to display results on the result page. # recommender/views.py

class MovieData:
    def __init__(self, movie_title, imdb_rating, rtm_rating, release_year):
        self.movie_title = movie_title
        self.imdb_rating = imdb_rating
        self.rtm_rating = rtm_rating
        self.release_year = release_year
    def __str__(self) -> str:
        return f"{self.movie_title} {self.imdb_rating} {self.rtm_rating} {self.release_year}"

def _merge_movie_lists(imdb_movie_list, rtm_movie_list, form: MovieSearchForm):
    merged_movie_list = []
    movie_name_movie_year_to_movie_data = defaultdict(lambda: {})
    for imdb_movie in imdb_movie_list:
        movie_name = imdb_movie.movie_title
        imdb_rating = imdb_movie.rating
        rtm_rating = None
        release_year = imdb_movie.release_date.year
        movie_name_movie_year_to_movie_data[movie_name.lower()][release_year] = MovieData(movie_name, imdb_rating, rtm_rating, release_year)
    for rtm_movie in rtm_movie_list:
        movie_name = rtm_movie.movie_title
        rtm_rating = rtm_movie.rating
        imdb_rating = None
        release_year = rtm_movie.release_year
        existing_object = movie_name_movie_year_to_movie_data[movie_name.lower()].get(release_year, None)
        if not existing_object:
            movie_name_movie_year_to_movie_data[movie_name.lower()][release_year] = MovieData(movie_name, imdb_rating, rtm_rating, release_year)
        else:
            existing_object.rtm_rating = rtm_rating
            movie_name_movie_year_to_movie_data[movie_name.lower()][release_year] = existing_object
    for movie_year_to_movie_data in movie_name_movie_year_to_movie_data.values():
        for movie_data in movie_year_to_movie_data.values():
            if not form.cleaned_data['include_empty_rotten_tomato_ratings'] and movie_data.rtm_rating is None:
                continue
            if not form.cleaned_data['include_empty_imdb_ratings'] and movie_data.imdb_rating is None:
                continue
            merged_movie_list.append(movie_data)
    return merged_movie_list This code might look overwhelming at first sight! But let us break this down to better understand this. (1) We first create a map from the movie name and movie release year to actual movie data. (2) We fill this map for IMDB movie data first. (3) Next, we try to fill this map for Rotten Tomato movie data. (4) Once the map is populated with data from both lists, we filter the entries in this map based on the values provided in the form by the user. (5) We finally return the merged list. Final Result Okay… with all that work done, What do we have? We can try searching for movies based on two different ratings, and our app WORKS! With all the methods described in this article, you can add many more such ratings. It is all up to the imagination of readers! Code For This Series All code created as a part of this series will be available on GitHub repository. The final result of this part is contained in branch named . this this Part2 It is always a good idea to cross-check your code with the one present on the repo to check if something is wrong. Concluding Remarks With this, we come to the end of the second part of this three-part series. In the next part, we will automate the process of data delivery so that our application stays up to date with all the new data. See you next time… Till then, Happy Learning! 🙂

Flix-Finder: Building a Movie Recommendation App With Django and Bright Data [Part 2/3]

Stories, News & Mentions about BRIGHT

Bright’s Stories on HackerNoon