introduction to data science assignment 4

• Stack Overflow for Teams Where developers & technologists share private knowledge with coworkers
• Advertising & Talent Reach devs & technologists worldwide about your product, service or employer brand
• OverflowAI GenAI features for Teams
• OverflowAPI Train & fine-tune LLMs
• Labs The future of collective knowledge sharing
• About the company Visit the blog

Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Get early access and see previews of new features.

"AssertionError: Q2: Your answer is not close enough to ours to be correct."(Introduction to data science in python Coursera)

I need some assistance with my Assignment 4 for Introduction to Data Science in Python.

Question 2 of assignment 4.

For this question, calculate the win/loss ratio's correlation with the population of the city it is in for the NBA using 2018 data.

When I submit the assignment this is the error i get

You have failed this test due to an error. The traceback has been removed because it may contain hidden tests. This is the exception that was thrown: AssertionError: Q2: Your answer is not close enough to ours to be correct.

The answer I get is

-0.16087318003059875

I have a feeling the answer must be around

-0.1763635064218294

since that is what the other online resources show me for people who have taken part in the test,

Here is my Dataframe output:

This is my solution for question 2 run in Jupyter notebook

I am not so sure how to go with this since am sure i have worked out the assignment to my full capabilities, also below is my question 1 code just in case it cuts across.

And here is the question to assignment 4.

In this assignment you must read in a file of metropolitan regions and associated sports teams from assets/wikipedia_data.html and answer some questions about each metropolitan region. Each of these regions may have one or more teams from the "Big 4": NFL (football, in assets/nfl.csv), MLB (baseball, in assets/mlb.csv), NBA (basketball, in assets/nba.csv or NHL (hockey, in assets/nhl.csv). Please keep in mind that all questions are from the perspective of the metropolitan region, and that this file is the "source of authority" for the location of a given sports team. Thus teams which are commonly known by a different area (e.g. "Oakland Raiders") need to be mapped into the metropolitan region given (e.g. San Francisco Bay Area). This will require some human data understanding outside of the data you've been given (e.g. you will have to hand-code some names, and might need to google to find out where teams are)! For each sport I would like you to answer the question: what is the win/loss ratio's correlation with the population of the city it is in? Win/Loss ratio refers to the number of wins over the number of wins plus the number of losses. Remember that to calculate the correlation with pearsonr, so you are going to send in two ordered lists of values, the populations from the wikipedia_data.html file and the win/loss ratio for a given sport in the same order. Average the win/loss ratios for those cities which have multiple teams of a single sport. Each sport is worth an equal amount in this assignment (20%*4=80%) of the grade for this assignment. You should only use data from year 2018 for your analysis -- this is important! Notes Do not include data about the MLS or CFL in any of the work you are doing, we're only interested in the Big 4 in this assignment. I highly suggest that you first tackle the four correlation questions in order, as they are all similar and worth the majority of grades for this assignment. This is by design! It's fair game to talk with peers about high level strategy as well as the relationship between metropolitan areas and sports teams. However, do not post code solving aspects of the assignment (including such as dictionaries mapping areas to teams, or regexes which will clean up names). There may be more teams than the assert statements test, remember to collapse multiple teams in one city into a single value! As this assignment utilizes global variables in the skeleton code, to avoid having errors in your code you can either: You can place all of your code within the function definitions for all of the questions (other than import statements). You can create copies of all the global variables with the copy() method and proceed as usual.

Here is some sample data so one can understand how the data in the CSV files looks like in Jupyter Notebook:

• jupyter-notebook

Know someone who can answer? Share a link to this question via email , Twitter , or Facebook .

Your answer.

Reminder: Answers generated by artificial intelligence tools are not allowed on Stack Overflow. Learn more

Sign up or log in

Post as a guest.

Required, but never shown

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy .

Browse other questions tagged python pandas dataframe numpy jupyter-notebook or ask your own question .

• The Overflow Blog
• Detecting errors in AI-generated code
• Featured on Meta
• User activation: Learnings and opportunities
• Preventing unauthorized automated access to the network
• Announcing the new Staging Ground Reviewer Stats Widget

Hot Network Questions

• Which design is better
• "Some" depicted
• What is the utility of the Tietze extension theorem?
• Can Inductors be thought of as storing voltage?
• Email from Deutsche Bahn about a timetable change - what do I need to do?
• Replacing "shall not", "shall be", "shall" etc. with "must" or other more imperative words
• Could a civilisation develop spaceflight by artillery before developing orbit-capable rockets?
• Can you perceive when a creature "starts its turn"?
• He worked in the field during most of the day
• What will the pressure of accelerated air be relative to the pressure that accelerated it?
• Hochschild cohomology and differential operators
• How can I connect heavy-gauge wire to a 20A breaker?
• The 12th Amendment: what if the presidential and vice-presidential candidates are from the same state?
• BSS138 level shifter - blocking current flow when high-side supply is not connected?
• Will running a dehumidifier in a basement impact room temperature?
• Is Wild Shape affected by Moonbeam?
• Middle path of buddha
• How does NASA calculate trajectory at planetary atmosphere entry
• Change style of True/False everywhere
• Why do you even need a heatshield - why not just cool the re-entry surfaces from inside?
• Can artistic depictions of crime (especially violence) be used as evidence?
• Fundamental group of the smooth locus of a normal algebraic surface is a quotient of that of a Zariski open subset
• How to implement a "scanner" effect on Linux to fix documents with varying darkness of background?
• Where did the baseball term "lace" come from?

Introduction to Data Science with Python

Learn python for data analysis.

Join Harvard University Instructor Pavlos Protopapas in this online course to learn how to use Python to harness and analyze data.

What You'll Learn

Every single minute, computers across the world collect millions of gigabytes of data. What can you do to make sense of this mountain of data? How do data scientists use this data for the applications that power our modern world?

Data science is an ever-evolving field, using algorithms and scientific methods to parse complex data sets. Data scientists use a range of programming languages, such as Python and R, to harness and analyze data. This course focuses on using Python in data science. By the end of the course, you’ll have a fundamental understanding of machine learning models and basic concepts around Machine Learning (ML) and Artificial Intelligence (AI). 

Using Python, learners will study regression models (Linear, Multilinear, and Polynomial) and classification models (kNN, Logistic), utilizing popular libraries such as sklearn, Pandas, matplotlib, and numPy. The course will cover key concepts of machine learning such as: picking the right complexity, preventing overfitting, regularization, assessing uncertainty, weighing trade-offs, and model evaluation. Participation in this course will build your confidence in using Python, preparing you for more advanced study in Machine Learning (ML) and Artificial Intelligence (AI), and advancement in your career.   Learners must have a minimum baseline of programming knowledge (preferably in Python) and statistics in order to be successful in this course. Python prerequisites can be met with an introductory Python course offered through CS50’s Introduction to Programming with Python , and statistics prerequisites can be met via Fat Chance or with Stat110 offered through HarvardX.

The course will be delivered via edX and connect learners around the world. By the end of the course, participants will learn:

• Gain hands-on experience and practice using Python to solve real data science challenges
• Practice Python coding for modeling, statistics, and storytelling
• Utilize popular libraries such as Pandas, numPy, matplotlib, and SKLearn
• Run basic machine learning models using Python, evaluate how those models are performing, and apply those models to real-world problems
• Build a foundation for the use of Python in machine learning and artificial intelligence, preparing you for future Python study

Your Instructor

Pavlos Protopapas is the Scientific Program Director of the Institute for Applied Computational Science(IACS) at the Harvard John A. Paulson School of Engineering and Applied Sciences. He has had a long and distinguished career as a scientist and data science educator, and currently teaches the CS109 course series for basic and advanced data science at Harvard University, as well as the capstone course (industry-sponsored data science projects) for the IACS master’s program at Harvard. Pavlos has a Ph.D in theoretical physics from the University of Pennsylvania and has focused recently on the use of machine learning and AI in astronomy, and computer science. He was Deputy Director of the National Expandable Clusters Program (NSCP) at the University of Pennsylvania, and was instrumental in creating the Initiative in Innovative Computing (IIC) at Harvard. Pavlos has taught multiple courses on machine learning and computational science at Harvard, and at summer schools, and at programs internationally.

Course Overview

• Linear Regression
• Multiple and Polynomial Regression
• Model Selection and Cross-Validation
• Bias, Variance, and Hyperparameters
• Classification and Logistic Regression
• Multi-logstic Regression and Missingness
• Bootstrap, Confidence Intervals, and Hypothesis Testing
• Capstone Project

Ways to take this course

When you enroll in this course, you will have the option of pursuing a Verified Certificate or Auditing the Course.

A Verified Certificate costs $299 and provides unlimited access to full course materials, activities, tests, and forums. At the end of the course, learners who earn a passing grade can receive a certificate. 

Alternatively, learners can Audit the course for free and have access to select course material, activities, tests, and forums.  Please note that this track does not offer a certificate for learners who earn a passing grade.

Related Courses

Data science professional certificate.

The HarvardX Data Science program prepares you with the necessary knowledge base and useful skills to tackle real-world data analysis challenges.

Machine Learning and AI with Python

Join Harvard University Instructor Pavlos Protopapas to learn how to use decision trees, the foundational algorithm for your understanding of machine learning and artificial intelligence.

Data Science for Business

Designed for managers, this course provides a hands-on approach for demystifying the data science ecosystem and making you a more conscientious consumer of information.

Introduction to Data Science I & II

Introduction, introduction #.

Dan L. Nicolae , Michael J. Franklin , Amanda R. Kube Jotte , Evelyn Campbell, Susanna Lange, Will Trimble, and Jesse London

Forthcoming…

Acknowledgements #

Jupyter Books was originally created by Sam Lau and Chris Holdgraf with support of the UC Berkeley Data Science Education Program and the Berkeley Institute for Data Science .

• Toggle navigation

Introduction to Data Science

Introduction to data science website, homework assignments and projects.

Table of Contents

Week 1: Introduction to Data Science

In three or four paragraphs:

• describe why you think the data science field is important. Use information you have learned from the videos, readings and lecture.
• describe one potential ethical problem related to data science, give an example
• describe one example of how data science affects your day-to-day life. Does it help or complicate your life? Why or why not.

Week 2: The Data Science Process

You are now informed about the data science process. In three to four paragraphs explain

• which steps are the most important in terms of ethical awareness and societal impact
• explain why you chose these steps
• include a current event in which one or multiple steps were ignored or compromised causing an ethical concern. Use newspaper articles, magazines, online news websites or any other legitimate and valid source to cite this example. Cite the news source that you found.

Week 3: Big Data

Big Data has many advantages however, it can also have negative impact. Choose an area where you think Big Data might cause more harm than good. Choose one area from the list below

• health care,
• personal loans
• immigration and citizenship
• incarceration
• social networks
• Describe how you think Big Data can be used to both help and hurt the area you have chosen.
• Find one current events that backs up your discussion regarding the negative impact of Big Data in the area of your choice. Use newspaper articles, magazines, online news websites or any other legitimate and valid source to cite this example. Cite the news source that you found.

Week 4: Managing Data

• Read the article Marr, Bernard. June 15, 2017. “ 3 Massive Big Data problems everyone should know about. ”
• Health care
• In a brief paragraph, describe the industry you chose. Cite your sources.
• In two or three paragraphs discuss how the three Big Data problems cited in the article “3 Massive Big Data problems everyone should know about” affect the industry you chose. Give examples. Use newspaper articles, magazines, online news websites

Week 5: Statistics for Data Science

Directions: For each question below, show all the work for full credit. You will not receive full credit for the question if steps are missing:

Question 1: Use the following data set for question 1

82, 66, 70, 59, 90, 78, 76, 95, 99, 84, 88, 76, 82, 81, 91, 64, 79, 76, 85, 90

• Find the Mean (10pts)
• Find the Median (10pts)
• Find the Mode (10pts)
• Find the Interquartile range (20pts)

Question 2:

• (20pts) Calculate the variance for the following data set: 10, 15, 5, 12, 20
• (25 pts) Find the linear regression for the following data set: x: 0,1,2,3,4 y: 2,3,5,4,6 Show all the steps required to show the linear regression.
• (5pts) Use the linear regression calculator   to check your answer from (b). Take a screen shot of the entire output from the calculator and include it in your homework for credit.

Week 6: Probability for Data Science

Part 1: (Exercise 1 and 2 below)

Exercise 1:

• 60% of boys play football
• 36% of boys play ice hockey
• 40% of boys that play football also play ice hockey
• What percent of those that play ice hockey also play football?

Exercise 2:

• 40% of the girls like music
• 24% of the girls like dance
• 30% of those that like music also like dance
• What percent of those that like dance also like music?

Part 2: In two paragraphs, explain why Bayes theorem is important in the Data Science field and how it is used. Cite your sources.

Week 7: Natural Language Processing

Natural Language Processing is used in our everyday lives. Some examples include; Google translate, chatbots on various websites, Siri or Alexa or Google Assist, auto-correct or auto-complete in software and search engines, relevant results on search pages, voice text messaging and many others. In four or five paragraphs;

• Choose one such example of NLP use
• Describe the example you chose: explain its features, its audience or users, and its benefits
• Find a current event in which the example of your choice had a negative impact. Explain what occurred and discuss the negative impact. Use newspaper articles, magazines, online news websites or any other legitimate and valid source to cite this example. Cite the news source that you found.
• Did the negative impact change the way you think about this NLP example? Why or why not

Week 8 and 9: Data Mining

• Using the knowledge you have gained from the lecture, readings and videos, choose an industry of your liking example: finance, healthcare, education, entertainment
• In one paragraph give a brief description of the industry
• Discuss how data mining is used in this industry
• Discuss how data mining for this industry can result in ethical concerns
• Discuss one current event in which the ethical concerns was realized. Use newspaper articles, magazines, online news websites or any other legitimate and valid source to cite this example. Cite the news source that you found.

Week 10 and 11: Machine Learning

After reviewing the lecture, readings and videos, use the following three Machine Learning tools.

• Machine Learning for Kids
• Quick, Draw
• Teachable Machine
• identify the target audience
• discuss the use of this tool by the target audience
• identify the tool’s benefits and drawbacks
• Predictive analytic
• Descriptive analytic
• Supervised learning
• Unsupervised learning
• Reinforcement learning
• Neural networks/deep learning

Week 12: Machine Learning Process

Read the article Ricci, Shaun. September 28, 2017. “ 4 Transformative Benefits of AI In Hiring ” Ideal 

• There are many advantages to implementing hiring models, however, there are also many disadvantages.
• Find one current event in which a hiring algorithm was used and produced biased or negative results. Use newspaper articles, magazines, online news websites or any other legitimate and valid source to cite this example. Cite the news source that you found.

Week 13: Data Visualization

There are many advantages to presenting big data visually using data visualization methods. However, there are also some disadvantages. Read the following two short articles:

• Read the article Kakande, Arthur. February 12. “ What’s in a chart? A Step-by-Step guide to Identifying Misinformation in Data Visualization. ” Medium
• Read the short web page Foley, Katherine Ellen. June 25, 2020. “ How bad Covid-19 data visualizations mislead the public. ” Quartz
• Research a current event which highlights the results of misinformation based on data visualization. Explain how the data visualization method failed in presenting accurate information. Use newspaper articles, magazines, online news websites or any other legitimate and valid source to cite this example. Cite the news source that you found.

Week 14: Data Science and Ethics

Watch the short KMOV St. Louis news video on YouTube regarding the installation of smart street lights in the St. Louis downtown area. (Zotos, Alexis. January 8, 2020. “’ Smart’ street lights being installed around downtown in hopes to increase safety. ”)

• What are the benefits associated with using these street lights?
• Explain how these street lights are an example of data science in sue. Be specific.

What are the ethical implications of using these types of street lights? Back up your argument with information found in newspaper articles, magazines, online news websites or any other legitimate and valid source. Cite the news source that you found

Project 1 [tba]

Project 2 [tba], project 3 [tba], the openlab at city tech: a place to learn, work, and share.

The OpenLab is an open-source, digital platform designed to support teaching and learning at City Tech (New York City College of Technology), and to promote student and faculty engagement in the intellectual and social life of the college community.

New York City College of Technology | City University of New York

Accessibility

Our goal is to make the OpenLab accessible for all users.

Learn more about accessibility on the OpenLab

Creative Commons

• - Attribution
• - NonCommercial
• - ShareAlike

© New York City College of Technology | City University of New York

Introduction to Data Science

A Python Approach to Concepts, Techniques and Applications

• © 2024
• Latest edition
• Laura Igual 0 ,
• Santi Seguí 1

Departament de Matemàtiques i Informàtica, Universitat de Barcelona, Barcelona, Spain

You can also search for this author in PubMed   Google Scholar

• Describes tools and techniques that demystify data science
• Discusses Python extensions, techniques and modules to perform statistical analysis and machine learning
• Includes case studies, and supplies code examples and data at an associated website

Part of the book series: Undergraduate Topics in Computer Science (UTICS)

19k Accesses

3 Citations

This is a preview of subscription content, log in via an institution to check access.

Access this book

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as EPUB and PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Other ways to access

Licence this eBook for your library

Institutional subscriptions

About this book

This accessible and classroom-tested textbook/reference presents an introduction to the fundamentals of the interdisciplinary field of data science. The coverage spans key concepts from statistics, machine/deep learning and responsible data science, useful techniques for network analysis and natural language processing, and practical applications of data science such as recommender systems or sentiment analysis. 

Topics and features:  

• Provides numerous practical case studies using real-world data throughout the book 
• Supports understanding through hands-on experience of solving data science problems using Python 
• Describes concepts, techniques and tools for statistical analysis, machine learning, graph analysis, natural language processing, deep learning and responsible data science
• Reviews a range of applications of data science, including recommender systems and sentiment analysis of text data 
• Provides supplementary code resources and data at an associated website 

This practically-focused textbook provides an ideal introduction to the field for upper-tier undergraduate and beginning graduate students from computer science, mathematics, statistics, and other technical disciplines. The work is also eminently suitable for professionals on continuous education short courses, and to researchers following self-study courses.

• Data Science
• Parallel Computing
• Python Programming

Statistical Inference

• Graph Analysis

Table of contents (12 chapters)

Front matter, introduction to data science.

Laura Igual, Santi Seguí

Data Science Tools

• Eloi Puertas

Descriptive Statistics

Supervised learning, regression analysis, unsupervised learning, network analysis, recommender systems, basics of natural language processing, deep learning, responsible data science, back matter, authors and affiliations, about the authors.

Dr. Laura Igual  is an Associate Professor at the Departament de Matemàtiques i Informàtica, Universitat de Barcelona, Spain.  Dr. Santi Seguí  is an Associate Professor at the same institution.

The authors wish to mention that some chapters were co-written by Jordi Vitrià, Eloi Puertas, Petia Radeva, Oriol Pujol, Sergio Escalera.

Bibliographic Information

Book Title : Introduction to Data Science

Book Subtitle : A Python Approach to Concepts, Techniques and Applications

Authors : Laura Igual, Santi Seguí

Series Title : Undergraduate Topics in Computer Science

DOI : https://doi.org/10.1007/978-3-031-48956-3

Publisher : Springer Cham

eBook Packages : Computer Science , Computer Science (R0)

Copyright Information : The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2024

Softcover ISBN : 978-3-031-48955-6 Published: 13 April 2024

eBook ISBN : 978-3-031-48956-3 Published: 12 April 2024

Series ISSN : 1863-7310

Series E-ISSN : 2197-1781

Edition Number : 2

Number of Pages : XIV, 246

Number of Illustrations : 4 b/w illustrations, 78 illustrations in colour

Topics : Data Structures and Information Theory , Artificial Intelligence , Data Mining and Knowledge Discovery , Python

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Member-only story

Hands-On Numerical Derivative with Python, from Zero to Hero

Here’s everything you need to know (beyond the standard definition) to master the numerical derivative world.

Piero Paialunga

Towards Data Science

There is a legendary statement that you can find in at least one lab at every university and it goes like this:

Theory is when you know everything but nothing works. Practice is when everything works but no one knows why. In this lab, we combine theory and practice: nothing works and nobody knows why

I find this sentence so relatable in the data science world. I say this because data science starts as a mathematical problem ( theory ): you need to minimize a loss function. Nonetheless, when you get to real life (experiment/lab) things start to get very messy and your perfect theoretical world assumptions might not work anymore (they never do), and you don’t know why.

For example, take the concept of derivative . Everybody who deals with complex concepts of data science knows (or, even better, MUST know) what a derivative is. But then how do you apply the elegant and theoretical concept of derivative in real life , on a noisy signal, where you don’t have the analytic…

Written by Piero Paialunga

PhD in Aerospace Engineering at the University of Cincinnati. Machine Learning Engineer @ Gen Nine, Martial Artist, Coffee Drinker, from Italy.

Text to speech

Instantly share code, notes, and snippets.

shantanuatgit / Assignment3.py

• Download ZIP
• Star ( 0 ) 0 You must be signed in to star a gist
• Fork ( 0 ) 0 You must be signed in to fork a gist
• Embed Embed this gist in your website.
• Share Copy sharable link for this gist.
• Clone via HTTPS Clone using the web URL.
• Learn more about clone URLs
• Save shantanuatgit/2054ad91d1b502bae4a8965d6fb297e1 to your computer and use it in GitHub Desktop.