pyspark - Is there a way to do two stage least squares in spark?

Question

I'm trying to implement an instrumental variables regression approach (two-stage least squares 2SLS IV regression) in spark. I have a preference for sparklyr or SparkR but pyspark would be fine.

There is an example of IVregression in R here. But the ivreg function from the AER package doesn't exists (as far as I can tell) in spark.

There is a way to do this with two separate regressions, used in the above blog, but that will not produce correct standard errors.

Answer 1

等待大神答复