It’s been a busy week — it’s tough to keep up with all the cybersecurity news. We’ve collected some of the biggest cybersecurity stories from the week — from TechCrunch and afar — to keep you up to date with the latest hacks, privacy breaches and security stories you need to know. Facebook now says […]

I’m starting to get that thing where my iPhone XS screen feels super tiny when I switch back from the Fold to send a text message from my number. Someone recently asked me if I’m going to have trouble giving the device back to Samsung in a few days, and while the answer is a […]

The government of Sri Lanka has temporarily blocked access to several social media services following deadly explosions that ripped through the country, killing at least 207 people and injuring hundreds more. Eight bombings were reported, including during Easter services at three churches, on the holiest weekend of the Christian calendar. In a brief statement, the […]

Stop me if you’ve heard this one before. Samsung tries to deliver a big innovation and fails miserably. A big story this week on TechCrunch was that in the buildup to the release of the Samsung Galaxy Fold, potentially one of the weirdest, most innovative, most expensive phones shipped in the past decade, there are […]

Names, soccer players, musicians and fictional characters make up some of the worst passwords of the year, according to the U.K. government’s National Cyber Security Center. But nothing beats “123456” as the worst password of all. It’s no shock to any seasoned security pro. For years, the six-digit password has been donned the worst password […]

Technology shrinks the world, makes geography less relevant. People find kinship, common cause, and community on the Internet, across nations and sometimes even languages. When the Internet began to erupt, when its connections began to draw such people closer together, this was anticipated with great hope and excitement. And with reason. At their best, the […]

Editor’s Note: No one knows how to hire Hiring is the lifeblood of the world. Few people do truly singular work; instead, nearly every facet of our civilization is built by groups of humans (and increasingly machines) working in tandem. That presents quite the puzzle though: if teamwork is so critical to the functioning of, […]

In this project, we will create a web application that displays a random inspirational quote. The goal of this project is to learn about application factory and how to create templates in Flask. This is the second part of the “Getting started with Flask series”. In the first part, we learned how to create a basic Hello World application using Flask and run it locally. Installation and Setup First, let us install the dependencies and setup our project directory. Basically what we need to do is: Install Python. We will be using Python3 here. Create a directory for our project, say “1_inspirational_quotes”, and go inside the directory. mkdir 1_inspirational_quotescd 1_inspirational_quotes We will create a virtual environment for our project where we will install Flask and any other dependencies. So go ahead and create a virtual environment and activate it. python3 -m venv venv. venv/bin/activate Finally install Flask. pip3 install flask If you need more instructions on installation, refer to Flask installation guide or Getting started with Flask. Set up the Application Factory Now that we have setup our project directory, the next thing that we need to do is to create the Flask application, which is nothing but an instance of the Flask class. We could create the Flask instance globally, the way we did in Getting Started with Flask: Hello World. However, in this example, we will create it within a function. Application Factory is the term used to refer to the method inside which we will create our application (Flask instance). All sorts of configuration and setup required for the application will also be done within the application factory. Finally it will return the Flask application instance. So let us create a directory ‘quotes’ and add an __init__.py file to it. This will make the directory get treated as a Python package. mkdir quotes cd quotes touch __init__.py Then let us define our application factory in this file. ––> 1_inspirational_quotes/quotes/__init__.py from flask import Flask def create_app(): """ create_app is the application factory. """ # Create the app. app = Flask(__name__) # Add a route. @app.route('/') def home(): return 'Hello there! :)' # Return the app. return app Notes: The method create_app is the application factory.Within the application factory, we created a Flask instance, app, which is nothing but our application. Note that __name__ refers to the package name here, i.e., quotes. This is what will be used as our application name.Then we created a placeholder method, home, which will serve the content for our app page. For now, it just returns some string which will get displayed on our browser when we run the application.The decorator, @app.route, links the URL (/) to the method, home. Run the basic application In order to make sure that everything is set up correctly, let us run the application and see if it is working. First, let us set the FLASK_APP environment variable to be our application package name. This basically tells Flask which application to run. export FLASK_APP=quotes We will also set the environment variable FLASK_ENV to development so that: debug mode is turned on and the debugger is activated.the server will be restarted whenever we make a code change. We can make modifications to our code and simply refresh the browser to see the changes in effect. export FLASK_ENV=development Note: If you are on Windows, use set  instead of export. Now we are ready to run the application. So go ahead and run it using the flask command. You should see an output similar to the following. flask run * Serving Flask app "quotes" (lazy loading) * Environment: development * Debug mode: on * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit) * Restarting with stat * Debugger is active! * Debugger PIN: 150-101-403 Note: Make sure that you are running the command from the ‘1_inspirational_quotes’ directory and not ‘quotes’. Otherwise, you will see the error “flask.cli.NoAppException: Could not import “quotes.quotes”.” To see the app in action, go to http://127.0.0.1:5000/ on your browser. You should see our message displayed in it as shown in the following image. Awesome! Now let us start building our quotes app. Add a template Currently, our app just displays the string, “Hello there! :)” to the user. In this section, we will learn how to create a template that shows a random inspirational quote. Return HTML content from the application factory The simplest way to achieve this is to return the HTML code as a string instead of our hello world string as shown below: —> 1_inspirational_quotes/quotes/__init__.py from flask import Flask def create_app(): """ create_app is the application factory. """ # Create the app. app = Flask(__name__) # Add a route. @app.route('/') def home(): return ''' <html> <body> I find that the harder I work, the more luck I seem to have. – Thomas Jefferson </body> </html> ''' # Return the app. return app Now if you go to http://127.0.0.1:5000/, you should see the quote displayed on the screen: Even though this works perfectly fine, this is not the best approach to serve HTML content for our application. First of all, the code does not look clean. Second, as our application grows, modifying and maintaining the template within the application factory will be tedious. So we need to isolate our template from the application factory. Create a static HTML template file A template is a file that contains static data as well as placeholders for dynamic data. In this section, we will just be creating static HTML template that displays a quote to our user. In a later section, we will see how to make it dynamic. Within the quotes directory, let us add a directory to keep our templates and move our quotes template to a separate HTML file. mkdir templates touch templates/quotes.html Note that our template is stored within a directory named templates under the application directory, quotes. This is where Flask expects its templates by default. –>1_inspirational_quotes/quotes/templates/quotes.html <!doctype html> <html> <body> I find that the harder I work, the more luck I seem to have. – Thomas Jefferson <body> </html> Register the template with the application factory Now we need to modify our application factory such that this HTML file is served when users visit our web page. —> 1_inspirational_quotes/quotes/__init__.py from flask import Flask, render_template def create_app(): """ create_app is the application factory. """ # Create the app. app = Flask(__name__) # Add a route. @app.route('/') def home(): return render_template('quotes.html') # Return the app. return app Note how we introduced the method, render_template(). In this case, it takes our HTML file name and returns its contents. Later on, when we learn about serving dynamic content, we will learn more about rendering and how Flask uses Jinja for template rendering. Now if we go to http://127.0.0.1:5000/, we should see the quote displayed on the screen just as we saw earlier. Update the template to render quotes dynamically using Jinja Now that we have learned how to create a template and register it with the application factory, let us see how we can serve content dynamically. Right now our app just displays the same quote every time someone visits. Our goal is to dynamically update the quote by selecting one randomly from a set of quotes. First, let us go ahead and create a list of quotes. To keep things simple, we will be adding it in memory within the application factory. In a later post, we will explore how to use databases with Flask. —> 1_inspirational_quotes/quotes/__init__.py from flask import Flask, render_template import random def create_app(): """ create_app is the application factory. """ # Create the app. app = Flask(__name__) # Add a route. @app.route('/') def home(): sample_quotes = [ "I find that the harder I work, the more luck I seem to have. – Thomas Jefferson", "Success is the sum of small efforts, repeated day in and day out. – Robert Collier", "There are no shortcuts to any place worth going. – Beverly Sills", "The only place where success comes before work is in the dictionary. – Vidal Sassoon", "You don’t drown by falling in the water; you drown by staying there. – Ed Cole" ] # Select a random quote. selected_quote = random.choice(sample_quotes) # Pass the selected quote to the template. return render_template('quotes.html', quote=selected_quote) # Return the app. return app As you can see, now we are passing an additional parameter, quote, to the render_template function. Flask uses Jinja to render dynamic content in the template. With this change, the variable, quote, becomes available in the template, quotes.html. Now let us see how we can update the template file to make use of this variable. –>1_inspirational_quotes/quotes/templates/quotes.html <!doctype html> <html> <body> {{ quote }} <body> </html> Here, {{..}} is the delimiter used by Jinja to denote expressions which will be evaluated and rendered in the final HTML document. Now if we go to http://127.0.0.1:5000/ and keep refreshing the page, we should see a different random quote selected from the list every time. A demo is shown below: Add a stylesheet As of now, our app works, but it looks very plain. So now we will see how to add a simple stylesheet to it. In Flask, just like templates were expected to be in the templates directory by default, static files like CSS stylesheets are expected to be in static directory within the application folder. So go ahead and create the directory and add a CSS file style.css to it. mkdir static touch static/style.css —>1_inspirational_quotes/quotes/static/style.css body { background-color: black; background-image: url("background.jpg"); background-size:cover; } .quote_div { text-align: center; color: white; font-size: 30px; padding: 25px 5px; margin: 15% auto auto; } You can also add a background image and keep it under the static directory as shown above. Now let us modify the template to use the stylesheet. –>1_inspirational_quotes/quotes/templates/quotes.html <!doctype html> <html> <head> <link rel="stylesheet" href="/{{ url_for('static', filename='style.css') }}"> </head> <body> <div class="quote_div"> {{ quote }} </div> <body> </html> Now if we go to http://127.0.0.1:5000/, we will see a nicer app! A demo: Conclusion In case you want to browse through the code or download and try it out, the Github link for this project is here. In this post, we learned how to create a basic Flask application that serves dynamic data using a Jinja template. For more advanced lessons with projects, stay tuned and subscribe to our blog!

Hello there!, So you just got a shiny new Raspberry Pi. Well done and congrats! In this tutorial, we are going to look at how we can set up your Raspberry Pi and get it up and running. What you need to get started First and foremost we have to ensure that we have all the required items to get started. Here is a short list of items that are absolutely required to get the ball rolling. Raspberry Pi (buy from Amazon) (you guessed it right!)USB power adapter (buy from Amazon)(you could get away with a normal USB cable, but it is not recommended)micro SD card / SD card adapter (buy from Amazon)(at least 8 GB in size, 32 GB or more recommended. Hey it’s not that costly!). The given link points to a bundle that includes an SD card adapter.USB card reader (buy from Amazon)(you will need this for plugging the microSD card onto your computer to load the Operating System. If your PC/Mac has a supported card reader, that’s fine too!).USB keyboard / Mouse (buy from Amazon)(to get the OS setup properly during installation)HDMI cable (buy from Amazon)and a monitor (well, you need to see what you’re doing) Optional Ethernet cable / USB-WiFi adapter (buy from Amazon)– Unless you have a Raspberry Pi 3 model b+, this will come in handy to connect to the internet and update your software packages You can also get these as a bundle (buy from Amazon) or if you have some of the components lying around, then you could selectively get them as needed. Getting your OS ready Raspberry Pi is a miniature computer and just like your laptop and desktop PC, it needs to run an Operating System. Since Raspberry Pi runs an ARM-based processor (If you haven’t heard of ARM processors, they are low power processors commonly found in your mobile phones and tablets), we need to use a supporting version of OS. Luckily both the Raspberry Pi foundation and many of the Linux community members have created many Operating Systems that you can choose from to run on your board! In this tutorial, we are going to use NOOBS distribution which you can find here. Once you go to the above link, choose NOOBS, and you will be given two options, select NOOBS. This is a large file and it might take some time. Be patient! In the meanwhile, we need another piece of software that helps us format and transfer our OS to our microSD card. You can find the software here. Once the SDFormatter tool is installed, and the OS image is downloaded, we are ready to go! The first step is to unzip the NOOBS file that you have downloaded. You will get a NOOBS_v3_0_0 folder or something similar depending on your OS version. Open up the SDFormatter tool and insert your microSD card onto your PC. You can either use a USB microSD adapter like this or use the one that’s in your laptop / PC. On the SDFormatter tool select the drive that corresponds to your SD card. Make sure this is correct!. IMPORTANT: Please back up any data before formatting! Once the formatting is complete, you can copy the contents of the NOOB folder which would be something similar to what is shown below onto your formatted SD card. Connecting all pieces and booting up! All right, we’re almost there! Now we need to connect our Raspberry Pi to our peripherals and boot it up for the first time. Here, I have used a USB keyboard, a Wireless Logitech mouse that has a wireless USB adapter. I have plugged in a USB Wi-Fi adapter to connect to my network and an HDMI cable to connect to my monitor. The overall setup after setting up everything including power looks something like this. Setting up the OS Once we have the setup ready, let’s connect the board to the peripherals and power and boot it up. Initially, you will see a dialog box to select the operating system Select Raspbian Full and click the install button. The installation dialog will follow afterward. It will copy the files and install the OS. Once the OS is installed, the system will reboot and you will be welcomed with the new desktop!. Next, you will be prompted to set up your account password, keyboard profile, and timezone as well as connect to a Wi-Fi endpoint. That is pretty much it! Hello world in Python Fire up the terminal by going to the menu button on the top left and selecting the terminal. Open up a python shell by entering python This opens up a python shell, say hello world from your shiny new raspberry pi! print "Hello world!" Wrap up So that’s it! We’ve made it to the end and we have a Raspberry Pi up and running! Woohoo! Now is the fun part, which is all the fun things we could do with it. I’ll be doing some interesting projects that you could follow along on this blog going forward. Make sure to subscribe to thecodebits.com to receive updates. See you all soon!

Are you going to PyCon (or a similar conference)? Join me and Kenneth Retiz as we discuss how to make the most of PyCon and what makes it special for each of us.

These are the ten most rated questions at Stack Overflow last week.Between brackets: [question score / answers count]Build date: 2019-04-20 22:00:34 GMTWhy does Python start at index -1 (as opposed to 0) when indexing a list from the end? - [63/7]How to remove list items depending on predecessor in python - [11/7]How does Python know the values already stored in its memory? - [10/3]Century handling in Pandas - [9/5]How do I check if current code is part of a try-except-block? - [9/2]Python sets versus arrays - [9/1]Shared python generator - [8/1]Convert pandas column of lists into matrix representation (One Hot Encoding) - [6/2]Python Flask as Windows Service - [6/2]Count of values grouped per month, year - Pandas - [6/2]

This tutorial covers various ways to execute loops in python. Loops is an important concept of any programming language which performs iterations i.e. run specific code repeatedly until a certain condition is reached.1. For LoopLike R and C programming language, you can use for loop in Python. It is one of the most commonly used loop method to automate the repetitive tasks.How for loop works?Suppose you are asked to print sequence of numbers from 1 to 9, increment by 2.for i in range(1,10,2): print(i)Output 13579range(1,10,2) means starts from 1 and ends with 9 (excluding 10), increment by 2. Iteration over listThis section covers how to run for in loop on a list. mylist = [30,21,33,42,53,64,71,86,97,10]for i in mylist: print(i)Output 30213342536471869710Suppose you need to select every 3rd value of list.for i in mylist[::3]: print(i)Output 30427110mylist[::3] is equivalent to mylist[0::3] which follows this syntax style list[start:stop:step]Python Loop Explained with ExamplesExample 1 : Create a new list with only items from list that is between 0 and 10l1 = [100, 1, 10, 2, 3, 5, 8, 13, 21, 34, 55, 98]new = [] #Blank listfor i in l1: if i > 0 and i <= 10: new.append(i)newOutput: [1, 10, 2, 3, 5, 8]It can also be done via numpy package by creating list as numpy array. See the code below.import numpy as npk=np.array(l1)new=k[np.where(k<=10)]Example 2 : Check which alphabet (a-z) is mentioned in stringSuppose you have a string named k and you want to check which alphabet exists in the string k. k = "deepanshu"import stringfor n in string.ascii_lowercase: if n in k: print(n + ' exists in ' + k) else: print(n + ' does not exist in ' + k)string.ascii_lowercase returns 'abcdefghijklmnopqrstuvwxyz'.Practical Examples : for in loop in PythonCreate sample pandas data frame for illustrative purpose.import pandas as pdnp.random.seed(234)df = pd.DataFrame({"x1" : np.random.randint(low=1, high=100, size=10), "Month1" : np.random.normal(size=10), "Month2" : np.random.normal(size=10), "Month3" : np.random.normal(size=10), "price" : range(10) })df1. Multiple each month column by 1.2 for i in range(1,4): print(df["Month"+str(i)]*1.2)range(1,4) returns 1, 2 and 3. str( ) function is used to covert to string. "Month" + str(1) means Month1.2. Store computed columns in new data frameimport pandas as pdnewDF = pd.DataFrame()for i in range(1,4): data = pd.DataFrame(df["Month"+str(i)]*1.2) newDF=pd.concat([newDF,data], axis=1)pd.DataFrame( ) is used to create blank data frame. The concat() function from pandas package is used to concatenate two data frames.3. Check if value of x1 >= 50, multiply each month cost by price. Otherwise same as month.import pandas as pdimport numpy as npfor i in range(1,4): df['newcol'+str(i)] = np.where(df['x1'] >= 50, df['Month'+str(i)] * df['price'], df['Month'+str(i)])In this example, we are adding new columns named newcol1, newcol2 and newcol3. np.where(condition, value_if condition meets, value_if condition does not meet) is used to construct IF ELSE statement.4. Filter data frame by each unique value of a column and store it in a separate data framemydata = pd.DataFrame({"X1" : ["A","A","B","B","C"]})for name in mydata.X1.unique(): temp = pd.DataFrame(mydata[mydata.X1 == name]) exec('{} = temp'.format(name))The unique( ) function is used to calculate distinct values of a variable. The exec( ) function is used for dynamic execution of Python program. See the usage of string format( ) function below -s= "Your Input""i am {}".format(s)Output: 'i am Your Input'Loop Control StatementsLoop control statements change execution from its normal iteration. When execution leaves a scope, all automatic objects that were created in that scope are destroyed.Python supports the following control statements.Continue statementBreak statementContinue StatementWhen continue statement is executed, it skips the further code in the loop and continue iteration.In the code below, we are avoiding letters a and d to be printed. for n in "abcdef": if n =="a" or n =="d": continue print("letter :", n)letter : bletter : cletter : eletter : fBreak StatementWhen break statement runs, it breaks or stops the loop.In this program, when n is either c or d, loop stops executing. for n in "abcdef": if n =="c" or n =="d": break print("letter :", n)letter : aletter : bfor loop with else clauseUsing else clause with for loop is not common among python developers community.The else clause executes after the loop completes. It means that the loop did not encounter a break statement.The program below calculates factors for numbers between 2 to 10. Else clause returns numbers which have no factors and are therefore prime numbers:for k in range(2, 10): for y in range(2, k): if k % y == 0: print( k, '=', y, '*', round(k/y)) break else: print(k, 'is a prime number')2 is a prime number3 is a prime number4 = 2 * 25 is a prime number6 = 2 * 37 is a prime number8 = 2 * 49 = 3 * 3While LoopWhile loop is used to execute code repeatedly until a condition is met. And when the condition becomes false, the line immediately after the loop in program is executed. i = 1while i < 10: print(i) i += 2 #means i = i + 2 print("new i :", i)Output:1new i : 33new i : 55new i : 77new i : 99new i : 11While Loop with If-Else StatementIf-Else statement can be used along with While loop. See the program below -counter = 1 while (counter <= 5): if counter < 2: print("Less than 2") elif counter > 4: print("Greater than 4") else: print(">= 2 and <=4") counter += 1 About Author: Deepanshu founded ListenData with a simple objective - Make analytics easy to understand and follow. He has over 7 years of experience in data science and predictive modeling. During his tenure, he has worked with global clients in various domains.Let's Get Connected: LinkedIn

Pandas being one of the most popular package in Python is widely used for data manipulation. It is a very powerful and versatile package which makes data cleaning and wrangling much easier and pleasant.The Pandas library has a great contribution to the python community and it makes python as one of the top programming language for data science and analytics. It has become first choice of data analysts and scientists for data analysis and manipulation.Data Analysis with Python : Pandas Step by Step GuideWhy pandas?It has many functions which are the essence for data handling. In short, it can perform the following tasks for you -Create a structured data set similar to R's data frame and Excel spreadsheet.Reading data from various sources such as CSV, TXT, XLSX, SQL database, R etc.Selecting particular rows or columns from data setArranging data in ascending or descending orderFiltering data based on some conditionsSummarizing data by classification variableReshape data into wide or long formatTime series analysisMerging and concatenating two datasetsIterate over the rows of datasetWriting or Exporting data in CSV or Excel formatDatasets:In this tutorial we will use two datasets: 'income' and 'iris'.'income' data : This data contains the income of various states from 2002 to 2015. The dataset contains 51 observations and 16 variables. Download link'iris' data: It comprises of 150 observations with 5 variables. We have 3 species of flowers(50 flowers for each specie) and for all of them the sepal length and width and petal length and width are given. Download link Important pandas functions to rememberThe following is a list of common tasks along with pandas functions. Utility Functions Extract Column Names df.columns Select first 2 rows df.iloc[:2] Select first 2 columns df.iloc[:,:2] Select columns by name df.loc[:,["col1","col2"]] Select random no. of rows df.sample(n = 10) Select fraction of random rows df.sample(frac = 0.2) Rename the variables df.rename( ) Selecting a column as index df.set_index( ) Removing rows or columns df.drop( ) Sorting values df.sort_values( ) Grouping variables df.groupby( ) Filtering df.query( ) Finding the missing values df.isnull( ) Dropping the missing values df.dropna( ) Removing the duplicates df.drop_duplicates( ) Creating dummies pd.get_dummies( ) Ranking df.rank( ) Cumulative sum df.cumsum( ) Quantiles df.quantile( ) Selecting numeric variables df.select_dtypes( ) Concatenating two dataframes pd.concat() Merging on basis of common variable pd.merge( ) Importing pandas libraryYou need to import or load the Pandas library first in order to use it. By "Importing a library", it means loading it into the memory and then you can use it. Run the following code to import pandas library:import pandas as pdThe "pd" is an alias or abbreviation which will be used as a shortcut to access or call pandas functions. To access the functions from pandas library, you just need to type pd.function instead of  pandas.function every time you need to apply it.Importing DatasetTo read or import data from CSV file, you can use read_csv() function. In the function, you need to specify the file location of your CSV file.income = pd.read_csv("C:\\Users\\Hp\\Python\\Basics\\income.csv") Index State Y2002 Y2003 Y2004 Y2005 Y2006 Y2007 \0 A Alabama 1296530 1317711 1118631 1492583 1107408 1440134 1 A Alaska 1170302 1960378 1818085 1447852 1861639 1465841 2 A Arizona 1742027 1968140 1377583 1782199 1102568 1109382 3 A Arkansas 1485531 1994927 1119299 1947979 1669191 1801213 4 C California 1685349 1675807 1889570 1480280 1735069 1812546 Y2008 Y2009 Y2010 Y2011 Y2012 Y2013 Y2014 Y2015 0 1945229 1944173 1237582 1440756 1186741 1852841 1558906 1916661 1 1551826 1436541 1629616 1230866 1512804 1985302 1580394 1979143 2 1752886 1554330 1300521 1130709 1907284 1363279 1525866 1647724 3 1188104 1628980 1669295 1928238 1216675 1591896 1360959 1329341 4 1487315 1663809 1624509 1639670 1921845 1156536 1388461 1644607 Get Variable NamesBy using income.columnscommand, you can fetch the names of variables of a data frame.Index(['Index', 'State', 'Y2002', 'Y2003', 'Y2004', 'Y2005', 'Y2006', 'Y2007', 'Y2008', 'Y2009', 'Y2010', 'Y2011', 'Y2012', 'Y2013', 'Y2014', 'Y2015'], dtype='object')income.columns[0:2] returns first two column names 'Index', 'State'. In python, indexing starts from 0. Knowing the Variable typesYou can use the dataFrameName.dtypes command to extract the information of types of variables stored in the data frame.income.dtypes Index objectState objectY2002 int64Y2003 int64Y2004 int64Y2005 int64Y2006 int64Y2007 int64Y2008 int64Y2009 int64Y2010 int64Y2011 int64Y2012 int64Y2013 int64Y2014 int64Y2015 int64dtype: objectHere 'object' means strings or character variables. 'int64' refers to numeric variables (without decimals).To see the variable type of one variable (let's say "State") instead of all the variables, you can use the command below -income['State'].dtypesIt returns dtype('O'). In this case, 'O' refers to object i.e. type of variable as character.Changing the data typesY2008 is an integer. Suppose we want to convert it to float (numeric variable with decimals) we can write:income.Y2008 = income.Y2008.astype(float)income.dtypesIndex objectState objectY2002 int64Y2003 int64Y2004 int64Y2005 int64Y2006 int64Y2007 int64Y2008 float64Y2009 int64Y2010 int64Y2011 int64Y2012 int64Y2013 int64Y2014 int64Y2015 int64dtype: objectTo view the dimensions or shape of the dataincome.shape (51, 16)51 is the number of rows and 16 is the number of columns.You can also use shape[0] to see the number of rows (similar to nrow() in R) and shape[1] for number of columns (similar to ncol() in R). income.shape[0]income.shape[1]To view only some of the rowsBy default head( ) shows first 5 rows. If we want to see a specific number of rows we can mention it in the parenthesis. Similarly tail( ) function shows last 5 rows by default.income.head()income.head(2) #shows first 2 rows.income.tail() income.tail(2) #shows last 2 rowsAlternatively, any of the following commands can be used to fetch first five rows.income[0:5] income.iloc[0:5]Define Categorical VariableLike factors() function in R, we can include categorical variable in python using "category" dtype.s = pd.Series([1,2,3,1,2], dtype="category")s0 11 22 33 14 2dtype: categoryCategories (3, int64): [1, 2, 3]Extract Unique ValuesThe unique() function shows the unique levels or categories in the dataset.income.Index.unique()array(['A', 'C', 'D', ..., 'U', 'V', 'W'], dtype=object)The nunique( ) shows the number of unique values.income.Index.nunique()It returns 19 as index column contains distinct 19 values.Generate Cross Tabpd.crosstab( ) is used to create a bivariate frequency distribution. Here the bivariate frequency distribution is between Index and State columns.pd.crosstab(income.Index,income.State)Creating a frequency distributionincome.Index selects the 'Index' column of 'income' dataset and value_counts( ) creates a frequency distribution. By default ascending = False i.e. it will show the 'Index' having the maximum frequency on the top.income.Index.value_counts(ascending = True) F 1G 1U 1L 1H 1P 1R 1D 2T 2S 2V 2K 2O 3C 3I 4W 4A 4M 8N 8Name: Index, dtype: int64To draw the samplesincome.sample( ) is used to draw random samples from the dataset containing all the columns. Here n = 5 depicts we need 5 columns and frac = 0.1 tells that we need 10 percent of the data as my sample.income.sample(n = 5)income.sample(frac = 0.1)Selecting only a few of the columnsTo select only a specific columns we use either loc[ ] or iloc[ ] functions. The index or columns to be selected are passed as lists. "Index":"Y2008" denotes the that all the columns from Index to Y2008 are to be selected.Syntax of df.loc[  ]df.loc[row_index , column_index]income.loc[:,["Index","State","Y2008"]]income.loc[0:2,["Index","State","Y2008"]]  #Selecting rows with Index label 0 to 2 & columnsincome.loc[:,"Index":"Y2008"]  #Selecting consecutive columns#In the above command both Index and Y2008 are included.income.iloc[:,0:5]  #Columns from 1 to 5 are included. 6th column not includedDifference between loc and ilocloc considers rows (or columns) with particular labels from the index. Whereas iloc considers rows (or columns) at particular positions in the index so it only takes integers.x = pd.DataFrame({"var1" : np.arange(1,20,2)}, index=[9,8,7,6,10, 1, 2, 3, 4, 5]) var19 18 37 56 710 91 112 133 154 175 19iloc Codex.iloc[:3]Output: var19 18 37 5loc codex.loc[:3]Output: var19 18 37 56 710 91 112 133 15You can also use the following syntax to select specific variables.income[["Index","State","Y2008"]]Renaming the variablesWe create a dataframe 'data' for information of people and their respective zodiac signs.data = pd.DataFrame({"A" : ["John","Mary","Julia","Kenny","Henry"], "B" : ["Libra","Capricorn","Aries","Scorpio","Aquarius"]})data  A B0 John Libra1 Mary Capricorn2 Julia Aries3 Kenny Scorpio4 Henry AquariusIf all the columns are to be renamed then we can use data.columns and assign the list of new column names.#Renaming all the variables.data.columns = ['Names','Zodiac Signs'] Names Zodiac Signs0 John Libra1 Mary Capricorn2 Julia Aries3 Kenny Scorpio4 Henry AquariusIf only some of the variables are to be renamed then we can use rename( ) function where the new names are passed in the form of a dictionary.#Renaming only some of the variables.data.rename(columns = {"Names":"Cust_Name"},inplace = True) Cust_Name Zodiac Signs0 John Libra1 Mary Capricorn2 Julia Aries3 Kenny Scorpio4 Henry AquariusBy default in pandas inplace = False which means that no changes are made in the original dataset. Thus if we wish to alter the original dataset we need to define inplace = True.Suppose we want to replace only a particular character in the list of the column names then we can use str.replace( ) function. For example, renaming the variables which contain "Y" as "Year"income.columns = income.columns.str.replace('Y' , 'Year ')income.columnsIndex(['Index', 'State', 'Year 2002', 'Year 2003', 'Year 2004', 'Year 2005', 'Year 2006', 'Year 2007', 'Year 2008', 'Year 2009', 'Year 2010', 'Year 2011', 'Year 2012', 'Year 2013', 'Year 2014', 'Year 2015'], dtype='object')Setting one column in the data frame as the indexUsing set_index("column name") we can set the indices as that column and that column gets removed.income.set_index("Index",inplace = True)income.head()#Note that the indices have changed and Index column is now no more a columnincome.columnsincome.reset_index(inplace = True)income.head()reset_index( ) tells us that one should use the by default indices.Removing the columns and rowsTo drop a column we use drop( ) where the first argument is a list of columns to be removed. By default axis = 0 which means the operation should take place horizontally, row wise. To remove a column we need to set axis = 1.income.drop('Index',axis = 1)#Alternativelyincome.drop("Index",axis = "columns")income.drop(['Index','State'],axis = 1)income.drop(0,axis = 0)income.drop(0,axis = "index")income.drop([0,1,2,3],axis = 0) Also inplace = False by default thus no alterations are made in the original dataset.  axis = "columns"  and axis = "index" means the column and row(index) should be removed respectively.Sorting the dataTo sort the data sort_values( ) function is deployed. By default inplace = False and ascending = True.income.sort_values("State",ascending = False)income.sort_values("State",ascending = False,inplace = True)income.Y2006.sort_values() We have got duplicated for Index thus we need to sort the dataframe firstly by Index and then for each particular index we sort the values by Y2002income.sort_values(["Index","Y2002"]) Create new variablesUsing eval( ) arithmetic operations on various columns can be carried out in a dataset.income["difference"] = income.Y2008-income.Y2009#Alternativelyincome["difference2"] = income.eval("Y2008 - Y2009")income.head() Index State Y2002 Y2003 Y2004 Y2005 Y2006 Y2007 \0 A Alabama 1296530 1317711 1118631 1492583 1107408 1440134 1 A Alaska 1170302 1960378 1818085 1447852 1861639 1465841 2 A Arizona 1742027 1968140 1377583 1782199 1102568 1109382 3 A Arkansas 1485531 1994927 1119299 1947979 1669191 1801213 4 C California 1685349 1675807 1889570 1480280 1735069 1812546 Y2008 Y2009 Y2010 Y2011 Y2012 Y2013 Y2014 Y2015 \0 1945229.0 1944173 1237582 1440756 1186741 1852841 1558906 1916661 1 1551826.0 1436541 1629616 1230866 1512804 1985302 1580394 1979143 2 1752886.0 1554330 1300521 1130709 1907284 1363279 1525866 1647724 3 1188104.0 1628980 1669295 1928238 1216675 1591896 1360959 1329341 4 1487315.0 1663809 1624509 1639670 1921845 1156536 1388461 1644607 difference difference2 0 1056.0 1056.0 1 115285.0 115285.0 2 198556.0 198556.0 3 -440876.0 -440876.0 4 -176494.0 -176494.0 income.ratio = income.Y2008/income.Y2009The above command does not work, thus to create new columns we need to use square brackets.We can also use assign( ) function but this command does not make changes in the original data as there is no inplace parameter. Hence we need to save it in a new dataset.data = income.assign(ratio = (income.Y2008 / income.Y2009))data.head()Finding Descriptive Statisticsdescribe( ) is used to find some statistics like mean,minimum, quartiles etc. for numeric variables.income.describe() #for numeric variables Y2002 Y2003 Y2004 Y2005 Y2006 \count 5.100000e+01 5.100000e+01 5.100000e+01 5.100000e+01 5.100000e+01 mean 1.566034e+06 1.509193e+06 1.540555e+06 1.522064e+06 1.530969e+06 std 2.464425e+05 2.641092e+05 2.813872e+05 2.671748e+05 2.505603e+05 min 1.111437e+06 1.110625e+06 1.118631e+06 1.122030e+06 1.102568e+06 25% 1.374180e+06 1.292390e+06 1.268292e+06 1.267340e+06 1.337236e+06 50% 1.584734e+06 1.485909e+06 1.522230e+06 1.480280e+06 1.531641e+06 75% 1.776054e+06 1.686698e+06 1.808109e+06 1.778170e+06 1.732259e+06 max 1.983285e+06 1.994927e+06 1.979395e+06 1.990062e+06 1.985692e+06 Y2007 Y2008 Y2009 Y2010 Y2011 \count 5.100000e+01 5.100000e+01 5.100000e+01 5.100000e+01 5.100000e+01 mean 1.553219e+06 1.538398e+06 1.658519e+06 1.504108e+06 1.574968e+06 std 2.539575e+05 2.958132e+05 2.361854e+05 2.400771e+05 2.657216e+05 min 1.109382e+06 1.112765e+06 1.116168e+06 1.103794e+06 1.116203e+06 25% 1.322419e+06 1.254244e+06 1.553958e+06 1.328439e+06 1.371730e+06 50% 1.563062e+06 1.545621e+06 1.658551e+06 1.498662e+06 1.575533e+06 75% 1.780589e+06 1.779538e+06 1.857746e+06 1.639186e+06 1.807766e+06 max 1.983568e+06 1.990431e+06 1.993136e+06 1.999102e+06 1.992996e+06 Y2012 Y2013 Y2014 Y2015 count 5.100000e+01 5.100000e+01 5.100000e+01 5.100000e+01 mean 1.591135e+06 1.530078e+06 1.583360e+06 1.588297e+06 std 2.837675e+05 2.827299e+05 2.601554e+05 2.743807e+05 min 1.108281e+06 1.100990e+06 1.110394e+06 1.110655e+06 25% 1.360654e+06 1.285738e+06 1.385703e+06 1.372523e+06 50% 1.643855e+06 1.531212e+06 1.580394e+06 1.627508e+06 75% 1.866322e+06 1.725377e+06 1.791594e+06 1.848316e+06 max 1.988270e+06 1.994022e+06 1.990412e+06 1.996005e+06 For character or string variables, you can write include = ['object']. It will return total count, maximum occurring string and its frequencyincome.describe(include = ['object'])  #Only for strings / objectsTo find out specific descriptive statistics of each column of data frameincome.mean()income.median()income.agg(["mean","median"])Mean, median, maximum and minimum can be obtained for a particular column(s) as:income.Y2008.mean()income.Y2008.median()income.Y2008.min()income.loc[:,["Y2002","Y2008"]].max()Groupby functionTo group the data by a categorical variable we use groupby( ) function and hence we can do the operations on each category.income.groupby("Index").Y2008.min()income.groupby("Index")["Y2008","Y2010"].max()agg( ) function is used to find all the functions for a given variable.income.groupby("Index").Y2002.agg(["count","min","max","mean"])income.groupby("Index")["Y2002","Y2003"].agg(["count","min","max","mean"])The following command finds minimum and maximum values for Y2002 and only mean for Y2003income.groupby("Index").agg({"Y2002": ["min","max"],"Y2003" : "mean"}) Y2002 Y2003 min max meanIndex A 1170302 1742027 1810289.000C 1343824 1685349 1595708.000D 1111437 1330403 1631207.000F 1964626 1964626 1468852.000G 1929009 1929009 1541565.000H 1461570 1461570 1200280.000I 1353210 1776918 1536164.500K 1509054 1813878 1369773.000L 1584734 1584734 1110625.000M 1221316 1983285 1535717.625N 1395149 1885081 1382499.625O 1173918 1802132 1569934.000P 1320191 1320191 1446723.000R 1501744 1501744 1942942.000S 1159037 1631522 1477072.000T 1520591 1811867 1398343.000U 1771096 1771096 1195861.000V 1134317 1146902 1498122.500W 1677347 1977749 1521118.500FilteringTo filter only those rows which have Index as "A" we write:income[income.Index == "A"]#Alternativelyincome.loc[income.Index == "A",:] Index State Y2002 Y2003 Y2004 Y2005 Y2006 Y2007 \0 A Alabama 1296530 1317711 1118631 1492583 1107408 1440134 1 A Alaska 1170302 1960378 1818085 1447852 1861639 1465841 2 A Arizona 1742027 1968140 1377583 1782199 1102568 1109382 3 A Arkansas 1485531 1994927 1119299 1947979 1669191 1801213 Y2008 Y2009 Y2010 Y2011 Y2012 Y2013 Y2014 Y2015 0 1945229 1944173 1237582 1440756 1186741 1852841 1558906 1916661 1 1551826 1436541 1629616 1230866 1512804 1985302 1580394 1979143 2 1752886 1554330 1300521 1130709 1907284 1363279 1525866 1647724 3 1188104 1628980 1669295 1928238 1216675 1591896 1360959 1329341 To select the States having Index as "A":income.loc[income.Index == "A","State"]income.loc[income.Index == "A",:].StateTo filter the rows with Index as "A" and income for 2002 > 1500000"income.loc[(income.Index == "A") & (income.Y2002 > 1500000),:]To filter the rows with index either "A" or "W", we can use isin( ) function:income.loc[(income.Index == "A") | (income.Index == "W"),:]#Alternatively.income.loc[income.Index.isin(["A","W"]),:] Index State Y2002 Y2003 Y2004 Y2005 Y2006 Y2007 \0 A Alabama 1296530 1317711 1118631 1492583 1107408 1440134 1 A Alaska 1170302 1960378 1818085 1447852 1861639 1465841 2 A Arizona 1742027 1968140 1377583 1782199 1102568 1109382 3 A Arkansas 1485531 1994927 1119299 1947979 1669191 1801213 47 W Washington 1977749 1687136 1199490 1163092 1334864 1621989 48 W West Virginia 1677347 1380662 1176100 1888948 1922085 1740826 49 W Wisconsin 1788920 1518578 1289663 1436888 1251678 1721874 50 W Wyoming 1775190 1498098 1198212 1881688 1750527 1523124 Y2008 Y2009 Y2010 Y2011 Y2012 Y2013 Y2014 Y2015 0 1945229 1944173 1237582 1440756 1186741 1852841 1558906 1916661 1 1551826 1436541 1629616 1230866 1512804 1985302 1580394 1979143 2 1752886 1554330 1300521 1130709 1907284 1363279 1525866 1647724 3 1188104 1628980 1669295 1928238 1216675 1591896 1360959 1329341 47 1545621 1555554 1179331 1150089 1775787 1273834 1387428 1377341 48 1238174 1539322 1539603 1872519 1462137 1683127 1204344 1198791 49 1980167 1901394 1648755 1940943 1729177 1510119 1701650 1846238 50 1587602 1504455 1282142 1881814 1673668 1994022 1204029 1853858 Alternatively we can use query( ) function and write our filtering criteria:income.query('Y2002>1700000 & Y2003 > 1500000')Dealing with missing valuesWe create a new dataframe named 'crops' and to create a NaN value we use np.nan by importing numpy.import numpy as npmydata = {'Crop': ['Rice', 'Wheat', 'Barley', 'Maize'],        'Yield': [1010, 1025.2, 1404.2, 1251.7],        'cost' : [102, np.nan, 20, 68]}crops = pd.DataFrame(mydata)cropsisnull( ) returns True and notnull( ) returns False if the value is NaN.crops.isnull()  #same as is.na in Rcrops.notnull()  #opposite of previous command.crops.isnull().sum()  #No. of missing values.crops.cost.isnull() firstly subsets the 'cost' from the dataframe and returns a logical vector with isnull()crops[crops.cost.isnull()] #shows the rows with NAs.crops[crops.cost.isnull()].Crop #shows the rows with NAs in crops.Cropcrops[crops.cost.notnull()].Crop #shows the rows without NAs in crops.CropTo drop all the rows which have missing values in any rows we use dropna(how = "any") . By default inplace = False . If how = "all" means drop a row if all the elements in that row are missingcrops.dropna(how = "any").shapecrops.dropna(how = "all").shape  To remove NaNs if any of 'Yield' or'cost' are missing we use the subset parameter and pass a list:crops.dropna(subset = ['Yield',"cost"],how = 'any').shapecrops.dropna(subset = ['Yield',"cost"],how = 'all').shapeReplacing the missing values by "UNKNOWN" sub attribute in Column name.crops['cost'].fillna(value = "UNKNOWN",inplace = True)cropsDealing with duplicatesWe create a new dataframe comprising of items and their respective prices.data = pd.DataFrame({"Items" : ["TV","Washing Machine","Mobile","TV","TV","Washing Machine"], "Price" : [10000,50000,20000,10000,10000,40000]})data Items Price0 TV 100001 Washing Machine 500002 Mobile 200003 TV 100004 TV 100005 Washing Machine 40000duplicated() returns a logical vector returning True when encounters duplicated.data.loc[data.duplicated(),:]data.loc[data.duplicated(keep = "first"),:]By default keep = 'first' i.e. the first occurence is considered a unique value and its repetitions are considered as duplicates.If keep = "last" the last occurence is considered a unique value and all its repetitions are considered as duplicates.data.loc[data.duplicated(keep = "last"),:] #last entries are not there,indices have changed.If keep = "False" then it considers all the occurences of the repeated observations as duplicates.data.loc[data.duplicated(keep = False),:]  #all the duplicates, including unique are shown.To drop the duplicates drop_duplicates is used with default inplace = False, keep = 'first' or 'last' or 'False' have the respective meanings as in duplicated( )data.drop_duplicates(keep = "first")data.drop_duplicates(keep = "last")data.drop_duplicates(keep = False,inplace = True)  #by default inplace = FalsedataCreating dummiesNow we will consider the iris dataset. iris = pd.read_csv("C:\\Users\\Hp\\Desktop\\work\\Python\\Basics\\pandas\\iris.csv")iris.head() Sepal.Length Sepal.Width Petal.Length Petal.Width Species0 5.1 3.5 1.4 0.2 setosa1 4.9 3.0 1.4 0.2 setosa2 4.7 3.2 1.3 0.2 setosa3 4.6 3.1 1.5 0.2 setosa4 5.0 3.6 1.4 0.2 setosamap( ) function is used to match the values and replace them in the new series automatically created.iris["setosa"] = iris.Species.map({"setosa" : 1,"versicolor":0, "virginica" : 0})iris.head()To create dummies get_dummies( ) is used. iris.Species.prefix = "Species" adds a prefix ' Species' to the new series created.pd.get_dummies(iris.Species,prefix = "Species")pd.get_dummies(iris.Species,prefix = "Species").iloc[:,0:1]  #1 is not includedspecies_dummies = pd.get_dummies(iris.Species,prefix = "Species").iloc[:,0:]With concat( ) function we can join multiple series or dataframes. axis = 1 denotes that they should be joined columnwise.iris = pd.concat([iris,species_dummies],axis = 1)iris.head() Sepal.Length Sepal.Width Petal.Length Petal.Width Species \0 5.1 3.5 1.4 0.2 setosa 1 4.9 3.0 1.4 0.2 setosa 2 4.7 3.2 1.3 0.2 setosa 3 4.6 3.1 1.5 0.2 setosa 4 5.0 3.6 1.4 0.2 setosa Species_setosa Species_versicolor Species_virginica 0 1 0 0 1 1 0 0 2 1 0 0 3 1 0 0 4 1 0 0 It is usual that for a variable with 'n' categories we creat 'n-1' dummies, thus to drop the first 'dummy' column we write drop_first = Truepd.get_dummies(iris,columns = ["Species"],drop_first = True).head()Ranking To create a dataframe of all the ranks we use rank( )iris.rank() Ranking by a specific variableSuppose we want to rank the Sepal.Length for different species in ascending order:iris['Rank'] = iris.sort_values(['Sepal.Length'], ascending=[True]).groupby(['Species']).cumcount() + 1iris.head( )#Alternativelyiris['Rank2'] = iris['Sepal.Length'].groupby(iris["Species"]).rank(ascending=1)iris.head()Calculating the Cumulative sumUsing cumsum( ) function we can obtain the cumulative sumiris['cum_sum'] = iris["Sepal.Length"].cumsum()iris.head()Cumulative sum by a variableTo find the cumulative sum of sepal lengths for different species we use groupby( ) and then use cumsum( )iris["cumsum2"] = iris.groupby(["Species"])["Sepal.Length"].cumsum()iris.head()Calculating the percentiles.Various quantiles can be obtained by using quantile( )iris.quantile(0.5)iris.quantile([0.1,0.2,0.5])iris.quantile(0.55)if else in PythonWe create a new dataframe of students' name and their respective zodiac signs.students = pd.DataFrame({'Names': ['John','Mary','Henry','Augustus','Kenny'],                         'Zodiac Signs': ['Aquarius','Libra','Gemini','Pisces','Virgo']})def name(row): if row["Names"] in ["John","Henry"]: return "yes" else: return "no"students['flag'] = students.apply(name, axis=1)studentsFunctions in python are defined using the block keyword def , followed with the function's name as the block's name. apply( ) function applies function along rows or columns of dataframe.Note :If using simple 'if else' we need to take care of the indentation . Python does not involve curly braces for the loops and if else.Output Names Zodiac Signs flag0 John Aquarius yes1 Mary Libra no2 Henry Gemini yes3 Augustus Pisces no4 Kenny Virgo noAlternatively, By importing numpy we can use np.where. The first argument is the condition to be evaluated, 2nd argument is the value if condition is True and last argument defines the value if the condition evaluated returns False.import numpy as npstudents['flag'] = np.where(students['Names'].isin(['John','Henry']), 'yes', 'no')studentsMultiple Conditions : If Else-if Elsedef mname(row): if row["Names"] == "John" and row["Zodiac Signs"] == "Aquarius" : return "yellow" elif row["Names"] == "Mary" and row["Zodiac Signs"] == "Libra" : return "blue" elif row["Zodiac Signs"] == "Pisces" : return "blue" else: return "black"students['color'] = students.apply(mname, axis=1)studentsWe create a list of conditions and their respective values if evaluated True and use np.select where default value is the value if all the conditions is Falseconditions = [    (students['Names'] == 'John') & (students['Zodiac Signs'] == 'Aquarius'),    (students['Names'] == 'Mary') & (students['Zodiac Signs'] == 'Libra'),    (students['Zodiac Signs'] == 'Pisces')]choices = ['yellow', 'blue', 'purple']students['color'] = np.select(conditions, choices, default='black')students Names Zodiac Signs flag color0 John Aquarius yes yellow1 Mary Libra no blue2 Henry Gemini yes black3 Augustus Pisces no purple4 Kenny Virgo no blackSelect numeric or categorical columns onlyTo include numeric columns we use select_dtypes( ) data1 = iris.select_dtypes(include=[np.number])data1.head() _get_numeric_data also provides utility to select the numeric columns only.data3 = iris._get_numeric_data()data3.head(3) Sepal.Length Sepal.Width Petal.Length Petal.Width cum_sum cumsum20 5.1 3.5 1.4 0.2 5.1 5.11 4.9 3.0 1.4 0.2 10.0 10.02 4.7 3.2 1.3 0.2 14.7 14.7For selecting categorical variablesdata4 = iris.select_dtypes(include = ['object'])data4.head(2) Species0 setosa1 setosaConcatenatingWe create 2 dataframes containing the details of the students:students = pd.DataFrame({'Names': ['John','Mary','Henry','Augustus','Kenny'],                         'Zodiac Signs': ['Aquarius','Libra','Gemini','Pisces','Virgo']})students2 = pd.DataFrame({'Names': ['John','Mary','Henry','Augustus','Kenny'],                          'Marks' : [50,81,98,25,35]}) using pd.concat( ) function we can join the 2 dataframes:data = pd.concat([students,students2])  #by default axis = 0 Marks Names Zodiac Signs0 NaN John Aquarius1 NaN Mary Libra2 NaN Henry Gemini3 NaN Augustus Pisces4 NaN Kenny Virgo0 50.0 John NaN1 81.0 Mary NaN2 98.0 Henry NaN3 25.0 Augustus NaN4 35.0 Kenny NaNBy default axis = 0 thus the new dataframe will be added row-wise. If a column is not present then in one of the dataframes it creates NaNs. To join column wise we set axis = 1data = pd.concat([students,students2],axis = 1)data Names Zodiac Signs Marks Names0 John Aquarius 50 John1 Mary Libra 81 Mary2 Henry Gemini 98 Henry3 Augustus Pisces 25 Augustus4 Kenny Virgo 35 KennyUsing append function we can join the dataframes row-wisestudents.append(students2)  #for rowsAlternatively we can create a dictionary of the two data frames and can use pd.concat to join the dataframes row wiseclasses = {'x': students, 'y': students2} result = pd.concat(classes)result  Marks Names Zodiac Signsx 0 NaN John Aquarius 1 NaN Mary Libra 2 NaN Henry Gemini 3 NaN Augustus Pisces 4 NaN Kenny Virgoy 0 50.0 John NaN 1 81.0 Mary NaN 2 98.0 Henry NaN 3 25.0 Augustus NaN 4 35.0 Kenny NaNMerging or joining on the basis of common variable.We take 2 dataframes with different number of observations:students = pd.DataFrame({'Names': ['John','Mary','Henry','Maria'],                         'Zodiac Signs': ['Aquarius','Libra','Gemini','Capricorn']})students2 = pd.DataFrame({'Names': ['John','Mary','Henry','Augustus','Kenny'],                          'Marks' : [50,81,98,25,35]})Using pd.merge we can join the two dataframes. on = 'Names' denotes the common variable on the basis of which the dataframes are to be combined is 'Names'result = pd.merge(students, students2, on='Names')  #it only takes intersectionsresult Names Zodiac Signs Marks0 John Aquarius 501 Mary Libra 812 Henry Gemini 98 By default how = "inner" thus it takes only the common elements in both the dataframes. If you want all the elements in both the dataframes set how = "outer" result = pd.merge(students, students2, on='Names',how = "outer")  #it only takes unionsresult Names Zodiac Signs Marks0 John Aquarius 50.01 Mary Libra 81.02 Henry Gemini 98.03 Maria Capricorn NaN4 Augustus NaN 25.05 Kenny NaN 35.0To take only intersections and all the values in left df set how = 'left'result = pd.merge(students, students2, on='Names',how = "left")result Names Zodiac Signs Marks0 John Aquarius 50.01 Mary Libra 81.02 Henry Gemini 98.03 Maria Capricorn NaNSimilarly how = 'right' takes only intersections and all the values in right df.result = pd.merge(students, students2, on='Names',how = "right",indicator = True)result Names Zodiac Signs Marks _merge0 John Aquarius 50 both1 Mary Libra 81 both2 Henry Gemini 98 both3 Augustus NaN 25 right_only4 Kenny NaN 35 right_onlyindicator = True creates a column for indicating that whether the values are present in both the dataframes or either left or right dataframe. About Author: Deepanshu founded ListenData with a simple objective - Make analytics easy to understand and follow. He has over 7 years of experience in data science and predictive modeling. During his tenure, he has worked with global clients in various domains.Let's Get Connected: LinkedIn

imapautofiler applies user-defined rules to automatically organize messages on an IMAP server. What’s new in 1.8.0? use yaml safe loader drop python 3.5 and add 3.7 support perform substring matches without regard to case

Why Tencent Shouldn't Launch a Gaming Smartphone  Motley FoolLaunching another high-end gaming phone in China's tough smartphone market doesn't make much sense.

How to find the right headset for your enterprise VR project  VentureBeatWith the ongoing proliferation of VR headsets, you'll be certain to find The One that's a perfect fit for your company, right?

US CIA Claims Huawei Is Directly Funded By Chinese Gov In Latest War Of Words  Hot HardwareThe United States government and Chinese telecommunications company Huawei have a strained relationship. The US Department of Justice and various ...

Amazon issues even steeper markdowns on Apple's 11-inch iPad Pro, delivering record low prices  AppleInsiderThe Apple authorized reseller has dropped 2018 iPad Pro prices by an additional $25 to $50, with popular 11-inch iPad Pro models starting at an all-time low of ...

Sony Reveals Details about its PlayStation 5 Console which will Compete with Microsoft's Xbox  XDigitalNewsWe have known for quite some time now that Sony is developing its next-generation PlayStation console, which is tentatively called the PlayStation 5.

Market Sheet: Watching the Technicals for Restoration Hardware Holdings (RH)  Williams ReviewFocusing in on technical signals for Restoration Hardware Holdings (RH), we have noted that the short-term moving average Hilo channel is currently Sell.

The Xbox One S All-Digital Edition - why pay more for less?  TechRadarWhile the shift from physical media has changed the world of music, movies and PC gaming in recent years, console gaming has been a bit more stubborn.

Another week down! Watch Game of Thrones season 8, episode 2 from wherever you are today.

Apple is ready the iPhone 11 for September, according to the latest leaks, and it may include a triple-lens camera.

I ditched my smartphone for a few days to see if a tablet would work in its place.

The ultimate camera phone at a brilliant price - grab these Google Pixel 3 XL deals before they're gone.

Poor pricing and a lack of innovation make the Xbox One S All-Digital Edition an off-putting prospect.

The size and design of the 2019 iPhone models is now more certain than ever after the latest leaked images.

This new Now TV deal allows you to stream every episode of the older seasons of Game of Thrones too.