Numpy Tutorial - Numpy Tutorial Python - Numpy Tutorial For Beginners | asingleneuron
Let's start our journey with numpy. NUMPY is a python library, capable of doing array processing with an ease.
Numpy is capable of performing any kind of array operation with a lightening speed. This is much needed tool to accomplish day to day task which requires multi-dimensional array. Numpy is also useful for linear algebra.
As a data scientist we daily deal with arrays :)
In this journey we will see useful function what makes you day to day task easy.
Prerequisite: you should be familiar with Python.
Load numpy library:
First thing first, load the numpy library by importing it.
import numpy as np # for arrays and linear algebra
1D Array or Vector:
we can create numpy array by using np.array.
arr = np.array([1,2,3])
print(type(arr))
print(arr)
Above example is creation of numpy array with the list.
Class of numpy array is ndarray.
print("Elemment type: ",arr.dtype)
dtype describes the type of elements in the array. For integers elements, int64 is default type.
print("Shape of an array:",arr.shape)
arr.shape returns a tuple (# of rows, # of cols). In above example it returns a tuple of (3,) it means it has 3 rows and 1 column.
print("Size of an array:",arr.size)
arr.size returns the total number of elements in the array. In above example we have only 3 elements.
2D Array or Matrix:
matrix = np.array([[1,2,3], [4,5,6]])
print(matrix)
print("Element type:",matrix.dtype)
print("Number of dimension:",matrix.ndim)
print("Shape of matrix:",matrix.shape)
print("Size of matrix:",matrix.size)
Float / Char / String Matrix:
We can also create matrix or vector of different datatype like float, char and string.
f_matrix = np.array([[1.0,2.0], [3.0, 4.0]])
print(f_matrix.dtype)
c_matrix = np.array([['a','b'], ['c', 'd']])
print(c_matrix.dtype.name)
s_matrix = np.array([["hello", "Hi"], ["Google", "Facebook"]])
print(s_matrix.dtype.name)
Mixed datatype:
Can we create a numpy array which contains mixed datatype ?
mix = [1, 2.0]
print(mix)
Above list contains two elements one is integer and another is float.
In python, a list can have any object. Let's try to create a numpy array with the above list.
print(np.array(mix))
All elements of a numpy array must have same datatype.
So when numpy founds one element integer and another float. It converts integer into float and creates an array.
mix = [1, 2.0, '3']
print(mix)
print(np.array(mix))
In this example, we have three different datatype mixed together.
So numpy converts integer and float into string and creates an array.
Smart ways to create ndarray:
np.ones:
np.ones : np.ones function creates an array of given shape and fills all the element with 1.
only_ones = np.ones((2,3))
print(only_ones)
print(only_ones.dtype.name)
In the above example, we have created an 2D array which contains 2 rows and 3 columns filled with 1. Default dtype for np.ones is float64.
only_ones_of_integer = np.ones((2,3), dtype=np.int16)
print(only_ones_of_integer)
print(only_ones_of_integer.dtype.name)
specify datatype : we can specify the datatype of numpy array by using dtype. In above example, we have specified the datatype int16.
np.zeros:
np.zeros : np.zeros function creates an array of given shape and fills all the element with 0.
only_zeros = np.zeros((2,3))
print(only_zeros)
In above example, we created a 2D array (2rows, 3colums) filled with zeros.
np.arange generates the values including start and excluding stop by using step to generate next number.
By default start is 0, step is 1.
So for np.arange(5) , it generated numbers by including 0 and excluding 5 by adding 1 as a step.
In the above example, start is 10 and end is 50 and step is 5.
np.arange generated an array starting from 10 , excluding 50 by incrementing 5.
Can we generate numbers from 5 to 1 by using np.arange ?
In the above example, we are trying to generate numbers in reverse order from 5 to 1.
To achive this, we have given start as 5, end as 0 and step is -1. So after generating 5 , it will add the step to get the next number. But here we are using the -1, so 5 -1 = 4.
One more thing to notice here is we use 0 as an end, because np.arange excludes the end.
np.linspace generates evenly spaced numbers over specified Start and End as input.
Number of elements to be generated is 50 by default.
In above example, we are trying to generate 10 evenly spaced numbers from 5 to 10 .
vector is a random 1D array, that has a shape of 10 rows and 1 column.
Re-shaped vector into matrix by using reshape() function. Specifying the new shape of 2 rows and 5 columns.
For exammple : if we are not sure about how many columns, than we can pass -1.
So reshape(2,-1) convert vector into matrix that has 2 rows, reshape() equally divides the number of elements into 2 chunks and figures out the number of columns.
With same trick we can convert matrix(2D array) to vector(1D array).
As we know, matrix has 2 rows and 5 columns, so 2 * 5 = 10 , passing 10 rows and 1 columns as an input to reshape() function.
We can also use the -1 trick to convert multi-dimensional to 1D array.
We know that 1D array has only 1 column, we can fix that and pass rows as -1.
Another way to flatten the multi-dimensional array is reshape(-1).
arr[3:7] is a way of selecting sub part of an array.
np.random:
np.random.random : np.random.random function creates an array of given shape and fills all the element with random values from [0, 1].
random_arr = np.random.random((2,3))
print(random_arr)
np.arange:
np.arange: np.arange generates a sequence of numbers and returns a single dimensional array. It takes stepsize to generate next number.
seq = np.arange(5)
print(seq)
By default start is 0, step is 1.
So for np.arange(5) , it generated numbers by including 0 and excluding 5 by adding 1 as a step.
np.arange(10,50,5)
np.arange generated an array starting from 10 , excluding 50 by incrementing 5.
Can we generate numbers from 5 to 1 by using np.arange ?
np.arange(5,0,-1)
To achive this, we have given start as 5, end as 0 and step is -1. So after generating 5 , it will add the step to get the next number. But here we are using the -1, so 5 -1 = 4.
One more thing to notice here is we use 0 as an end, because np.arange excludes the end.
np.linspace:
np.linespace: np.linespace generates evenly spaced numbers over a specified interval.
seq = np.linspace(0,10)
print("Number of elements:",seq.size)
print()
print(seq)
Number of elements to be generated is 50 by default.
start = 5
end = 10
num_of_elements = 10
np.linspace(start, end, num_of_elements)
Play with the SHAPE of array:
In this section we will see the tricks of array re-shaping.
For example: convert 1D array to 2D array or convert 2D array to 1D.
vector = np.random.random((10,1))
print(vector.shape)
print(vector)
matrix = vector.reshape(2,5)
print(matrix.shape)
print(matrix)
matrix = vector.reshape(2,-1)
print(matrix.shape)
reshape() function also works with -1, we can pass -1 if we are not sure about that dimension.
For exammple : if we are not sure about how many columns, than we can pass -1.
So reshape(2,-1) convert vector into matrix that has 2 rows, reshape() equally divides the number of elements into 2 chunks and figures out the number of columns.
print(matrix)
print(matrix.reshape(10,1))
As we know, matrix has 2 rows and 5 columns, so 2 * 5 = 10 , passing 10 rows and 1 columns as an input to reshape() function.
print(matrix.reshape(-1,1))
We know that 1D array has only 1 column, we can fix that and pass rows as -1.
flatten_array = matrix.reshape(-1)
print(flatten_array.shape)
Another way to flatten the multi-dimensional array is reshape(-1).
print(flatten_array)
Array indexing Tricks:
Just a reminder , array index starts from 0 (zero).
arr = np.arange(1,11)
arr
print("Access the first element of an array :",arr[0])
print("Access the fifth element of an array :",arr[4])
print()
print("Access the last element of an array: ",arr[-1])
print("Access the second element of an array: ",arr[-2])
First element start with index 0, second with 1 and so on.
-ve index starts from the last element by starting -1, -2 and so on.start_index = 3
end_index = 7
print(arr[ start_index : end_index])
We need to specify the start and end (excluding) index. By default index steps by 1.
start_index = 0
end_index = 10
step_by = 2
print(arr[ start_index : end_index: step_by])
In above example we are finding the subset of array by passing starting index 0 and end index 10 and by stepping 2 index. So index will be 0, 2, 4, 6, 8.
matrix = np.arange(1,17).reshape(4,4)
print(matrix)
print("Element at first row and first column of matrix:",matrix[0,0])
print("Element at last row and last column of matrix:",matrix[3,3])
print("Second row of the matrix:")
matrix[1]
print("First to third rows of the matrix:")
matrix[0:3]
print("Second column of the matrix:")
matrix[:,1]
print("First to Third columns of the matrix:")
matrix[:,0:3]
print("Find elements of second and third column and second and third row of the matrix")
matrix[1:3, 1:3]
Boolean indexing or Masking:
matrix
mask = matrix > 10
mask
In above operation, we are executing a logical condition which returns True if element of matrix is greater than 10 else False. This condition will be executed for all the array elements.
As a result , we got a boolean array contains True / False.
matrix[mask]
matrix[mask] is the way of boolean indexing. It returns a sub-array of elements which satisfies the condition. So if mask array has True than the element of that index will be picked, else will drop from the resulting array.
One liner to achieve the same results:
matrix[matrix > 10]
Another Example:
Boolean indexing to find the sub-array of all the even numbers.
even_element_mask = matrix % 2 == 0
even_element_mask
matrix[even_element_mask]
Matrix Operations:
1. Transpose:
Matrix transpose is changing rows to columns and columns to rows.
matrix = np.arange(1,11).reshape(2,-1)
matrix
transposed_matrix = matrix.T
transposed_matrix
print("Matrix:",matrix.shape)
print("Transposed matrix: ",transposed_matrix.shape)
As we can see that rows of matrix becomes columns and columns becomes rows after applying transpose.
2. Element wise operations:
Element wise operations are those operations that gets applied to every element of the array.
matrix + 10
matrix - 10
matrix * matrix
All of the above examples are element-wise operations.
3. Dot product:
Matrices should satisfies the following condition:
1. Number of columns in the 1st matrix must be equal to the number of rows in 2nd matrix.
2. Dot product of M x N matrix with N x K matrix results M x K matrix.
print("Matrix:",matrix.shape)
print("Transposed matrix: ",transposed_matrix.shape)
matrix.dot(transposed_matrix)
4. inverse of matrix:
dummy= np.array([[1,2],[3,4]])
dummy
np.linalg.inv(dummy)
We can find the inverse of the Matrix by using np.linalg.inv().
Stacking and Splitting:
1. Horizontal Stacking:
Stacks array horizontally (column wise).
h_stacked = np.hstack([matrix, matrix])
h_stacked
In the above example, we stacked two matrix column wise. As a results we can see increased #of columns.
2. Vertical Stacking:
Stacks array vertically ( row wise).
v_stacked = np.vstack([matrix, matrix])
v_stacked
In the above example , we stacked two matrix row wise. As a result we can see increased #of rows.
3. Horizontal Splitting:
Splits array into multiple sub-arrays column wise (horizontally). We also need to specify the number of splits.
arr1, arr2 = np.hsplit(h_stacked, 2)
print("Array1:")
print(arr1)
print("\nArray2:")
print(arr2)
In the above example, we are splitting array column-wise into 2 sub-arrays.
4. Vertical Splitting:
Splits array into multiple sub-array row-wise (vertically). We also need to specify number of splits.
arr1, arr2 = np.vsplit(v_stacked, 2)
print("Array1:")
print(arr1)
print("\nArray2:")
print(arr2)
In above example, we are splitting an array into 2 sub-arrays row-wise.
Query ?
arr = np.array([[1,1,0,0],[1,1,0,0],[1,1,0,0],[1,1,0,0] ])
print(arr)
1. np.all :
np.all() test if all of the elements of any array along with given axis evaluates to True returns True, else False. It like logical AND operation.
By default axis is None, it will evaluate the condition on all the elements of the array.
np.all(arr==0)
In the above example , np.all() evaluates the condition arr==0 on all the elements of the array. As a result it returns False because not all the elements satisfies the condition.
np.all(arr==0, axis=0)
Now we have specified the axis=0 (columns wise) along with the condition. As a result we got False for columns 1 and 2 and True for column 3 and 4 because all the elements of column 3 and 4 has zero.
np.all(arr==0, axis=1)
Same query but now the axis=1 (row wise).
2. np.any :
np.any() test if any of the array elements of any array along with given axis evaluates to True returns True, else False. It like logical OR operation.
By default axis is None, it will evaluate the condition on all the elements of the array.
np.any(arr==0)
np.any(arr==0, axis=0)
np.any(arr==0, axis=1)
3. np.nonzero :
np.nonzero() retuns the number of indices of the elements that are non-zero. Returns two arrays, one for row indices another for column indices.
We can also specify the condition.
row_index, col_index = np.nonzero(arr)
print(row_index, col_index)
In above example, we have passed the array arr to np.nonzero(). It will check if an element is non-zero or not. If yes, returns the row and column indices of all the elements who satisfies.
Try to print the elements of returned indices to verify.
for r, c in zip(row_index, col_index):
print("arr[{0}][{1}] = {2}".format(r,c, arr[r][c]))
We can also pass any condition to np.nonzero(). Try to find all the zero elements.
row_index, col_index = np.nonzero(arr==0)
print(row_index, col_index)
for r, c in zip(row_index, col_index):
print("arr[{0}][{1}] = {2}".format(r,c, arr[r][c]))
4. np.where:
np.where() returns elements choosen from X or Y after evaluating the given condition. If X or Y is not given results will be same as np.nonzero().
np.where(arr==0)
In the above example we want to find the indices of all the elements which has a value zero. Above results are same as np.nonzero()
Let's try to use X or Y values.
np.where(arr==0, "zero", "non_zero")
Now are using "zero" and "non_zero" conditional values along with the condition. As a result we got a new array filled with specified values.
Values X ("zero") will be chosen for those elements which satisfies the condition , else value Y ("non-zero") will.
Not only scaler values, we can pass arrays also in place of X or Y. In that case element will be chosen from the same index.
np.where(arr==0, arr, -1)
Ordering:
data =[[11, 0, 3, 4], [34, 5, 1, 9], [-9, 5, 3, 6]]
X = np.array(data)
X
1. max:
**np.max()** returns the element of the array that has the maximum value. We can also specify the axis.
X.max() # maximum element of the array
X.max(axis=0) # Column-wise maximum elements
X.max(axis=1) # Row-wise maximum elements
2. min:
**np.min()** returns the element of the array that has the minimum value. We can also specify the axis.
X.min() # miniumn element of the array
X.min(axis=0) # Column-wise minimum elements of the array
X.min(axis=1) # Row-wise minimum elements of the array
3. np.sort:
np.sort() returns the sorted array in ascending order. We can perform the sorting along with axis also.
X
np.sort(X) # sort array elements in ascending order. By default it's sort the array row-wise
Trick to sort the elements in descending order.
-np.sort(-X) # sort array element in descending order.
Trick is first make all positive elements negative and all negative elements positive. np.sort() will sort all elements in ascending order. Finally reverse the sign of the sorted array.
np.sort(X, axis=0) # sort elements column-wise
np.sort(X, axis=1) #Sort elements row-wise
4. np.argmax:
np.argmax() returns the index of the maximum element.
X
X.reshape(-1) # Flattened array
np.argmax(X) # returns the index of the maximum element of the array. If axis is not given , it flattens the array first and finds the index.
np.argmax(X, axis=0) # indices of the maximum elements of the array column-wise
np.argmax(X, axis=1) # indices of the maximum elements of the array row-wise
5. np.argmin:
np.argmin() returns the index of the minimum elememt.
X
X.reshape(-1) # Flattened array
np.argmin(X) # returns the index of the minimum element of the array. If axis is not given , it flattens the array first and finds the index.
np.argmin(X, axis=0) # returns the indices of the minimum elements of the array column-wise
np.argmin(X, axis=1) # returns the indices of the minimum elements of the array row-wise
6. np.argsort:
np.argsort() returns the array of indices, after sorting it by ascending order.
X
np.argsort(X) # returns the array of indices after sorting the elements in ascending order.
[0, 2, 1, 3]])
Statistics :
X = np.arange(1, 11).reshape(5,2)
X
1. mean:
np.mean() computes the mean/average along with the specified axis. By default for the flattened array.
Formula :
np.mean(X) # returns mean of the elements on flattened array
np.mean(X, axis=0) # returns the mean of the elements column-wise
np.mean(X, axis=1) # returns the mean of the elements row-wise
2. median:
np.median() computes the median along with the specified axis. By default for the flattened array.
To find median, sort the array in ascending order first. Then return the middle element if array has odd number of elements. In case of even number of elements, return the average of two middle elements.
np.median(X) # returns median of the elements on flattened array
np.median(X, axis=0) # returns the median of the elements column-wise
np.median(X, axis=1) # returns the median of the elements row-wise
3. variance:
np.var() computes the variance along with the specified axis. By default if axis is not given, variance is calculated for the flattened array.
Formula:
np.var(X) # returns variance of the elements on flattened array
np.var(X, axis=0) # returns the variance of the elements column-wise
np.var(X, axis=1) # returns the variance of the elements row-wise
4. standard-deviation:
np.std() computes the stardard deviation along with the specified axis. By default if axis is not given, standard deviation is calculated for the flattened array.
Formula:
np.std(X) # returns standard-deviation of the elements on flattened array
np.std(X, axis=0) # returns standard-deviation of the elements column-wise
np.std(X, axis=1) # returns standard-deviation of the elements row-wise
You can download the code and make you hands dirty by using kaggle kernel .
Comments
Post a Comment