python-3.x - ImportError：无法从“ pyspark.sql”导入名称“ HiveContext”(ImportError: cannot import name 'HiveContext' from 'pyspark.sql')

Question

I am running pyspark in my PC (windows 10) but I can not import HiveContext:

(我在PC（Windows 10）中运行pyspark，但无法导入HiveContext：)

from pyspark.sql import HiveContext
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-25-e3ae767de910> in <module>
----> 1 from pyspark.sql import HiveContext

ImportError: cannot import name 'HiveContext' from 'pyspark.sql' (C:sparkspark-3.0.0-preview-bin-hadoop2.7pythonpysparksql\__init__.py)

The following code is also returning an exception:

(以下代码也返回异常：)

from pyspark.sql import SparkSession

spark = SparkSession 
.builder 
.appName("Python Spark create RDD example") 
.config("spark.some.config.option", "some-value") 
.getOrCreate()

# Create RDDs from an existing Hive repository
hive_ctx = spark.builder.enableHiveSupport()
hive_lines = hive_ctx.sql("SELECT name, age FROM users WHERE age > 18")

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-33-a4b672533a95> in <module>
      1 # Create RDDs from an existing Hive repository
      2 hive_ctx = spark.builder.enableHiveSupport()
----> 3 hive_lines = hive_ctx.sql("SELECT name, age FROM users WHERE age > 18")

AttributeError: 'Builder' object has no attribute 'sql'

as well as this:

(以及这个：)

# Create RDDs from an existing Hive repository
hive_ctx = spark.builder.enableHiveSupport(sc)
hive_lines = hive_ctx.sql("SELECT name, age FROM users WHERE age > 18")

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-34-883015e73112> in <module>
      1 # Create RDDs from an existing Hive repository
----> 2 hive_ctx = spark.builder.enableHiveSupport(sc)
      3 hive_lines = hive_ctx.sql("SELECT name, age FROM users WHERE age > 18")

TypeError: enableHiveSupport() takes 1 positional argument but 2 were given

How I should correct the code?

(我应该如何纠正代码？)

  ask by user8270077 translate from so

Answer 1

You're using the preview release of Spark 3.0.

(您正在使用Spark 3.0的预览版。)

According to the release notes , you should use SparkSession.builder.enableHiveSupport() .

(根据发行说明 ，您应该使用SparkSession.builder.enableHiveSupport() 。)

In Spark 3.0, the deprecated HiveContext class has been removed.

(在Spark 3.0中，已弃用的HiveContext类已被删除。)

Use SparkSession.builder.enableHiveSupport() instead.

(请改用SparkSession.builder.enableHiveSupport（）。)