pandas - python - cannot make corr work

Question

I'm struggling with getting a simple correlation done. I've tried all that was suggested under similar questions.

Here are the relevant parts of the code, the various attempts I've made and their results.

import numpy as np
import pandas as pd

try01 = data[['ESA Index_close_px', 'CCMP Index_close_px' ]].corr(method='pearson')

print (try01) 

Out:

Empty DataFrame
Columns: []
Index: []

---

try04 = data['ESA Index_close_px'][5:50].corr(data['CCMP Index_close_px'][5:50])
print (try04)

Out:

**AttributeError: 'float' object has no attribute 'sqrt'**

---

using numpy

try05 = np.corrcoef(data['ESA Index_close_px'],data['CCMP Index_close_px'])
print (try05)

Out:

AttributeError: 'float' object has no attribute 'sqrt'

---

converting the columns to lists

ESA_Index_close_px_list = list()
start_value = 1
end_value = len (data['ESA Index_close_px']) +1
for items in data['ESA Index_close_px']:
    ESA_Index_close_px_list.append(items)
    start_value = start_value+1    
    if start_value == end_value:
        break
    else:
        continue

CCMP_Index_close_px_list = list()
start_value = 1
end_value = len (data['CCMP Index_close_px']) +1
for items in data['CCMP Index_close_px']:
    CCMP_Index_close_px_list.append(items)
    start_value = start_value+1    
    if start_value == end_value:
        break
    else:
        continue

try06 = np.corrcoef(['ESA_Index_close_px_list','CCMP_Index_close_px_list'])
print (try06)

Out:

****TypeError: cannot perform reduce with flexible type****

---

Also tried .astype but not made any difference.

data['ESA Index_close_px'].astype(float)

data['CCMP Index_close_px'].astype(float)

Using Python 3.5, pandas 0.18.1 and numpy 1.11.1

Would really appreciate any suggestion.

**edit1:* Data is coming from an excel spreadsheet data = pd.read_excel('C:\Users\Ako\Desktop\ako_files\for_corr_??tool.xlsx') prior to the correlation attempts, there are only column renames and

data = data.drop(data.index[0]) 

to get rid of a line

regarding the types:

print (type (data['ESA Index_close_px']))



print (type (data['ESA Index_close_px'][1]))

Out:

**edit2* parts of the data:

print (data['ESA Index_close_px'][1:10])

print (data['CCMP Index_close_px'][1:10])

Out:

2        2137
3        2138
4        2132
5        2123
6        2127
7     2126.25
8      2131.5
9      2134.5
10       2159
Name: ESA Index_close_px, dtype: object
2     5241.83
3     5246.41
4     5243.84
5     5199.82
6     5214.16
7     5213.33
8     5239.02
9     5246.79
10    5328.67
Name: CCMP Index_close_px, dtype: object

Answer 1

Well, I've encountered the same problem today. try use .astype('float64') to help make the type correct.
data['ESA Index_close_px'][5:50].astype('float64').corr(data['CCMP Index_close_px'][5:50].astype('float64'))

This works well for me. Hope it can help you as well.