Wednesday, November 22, 2017

Connecting Zeppelin, Spark, and MongoDB

It took me a few hours to connect Zeppelin, Spark, and MongoDB.  I didn't find a solution to this problem online; thus the short entry.

First, I added a dependency to the MongoDB Connector for Spark in my Zeppelin notebook.


%dep
z.reset()
z.load("org.mongodb.spark:mongo-spark-connector_2.10:2.2.0")

%spark
import com.mongodb.spark._
import com.mongodb.spark.rdd.MongoRDD
val rdd = MongoSpark.load(sc)

This gave :

java.lang.IllegalArgumentException: Missing database name. Set via the 'spark.mongodb.input.uri' or 'spark.mongodb.input.database' property

Then, after realizing,  that you cannot dynamically reconfigure the SparkContext.  I used the GUI to set the property.


It is working well now!

rdd: com.mongodb.spark.rdd.MongoRDD[org.bson.Document] = MongoRDD[0] at RDD at MongoRDD.scala:47

18 comments:

  1. Great blog thanks for sharing Instagram and Facebook have provided an amazing place for new brands to grow and flourish. We can find the perfect niche for your brand on the best social media marketing in chennai.

    ReplyDelete
  2. Nice blog thanks for sharing Set up a aesthetic work environment that employees love to spend time in and relieve their stress. Your company needs the best corporate gardening service in Chennai and Karuna Nursery Gardens in happy to oblige you in the endeavour to make your infrastructure something worth flaunting about.

    ReplyDelete
  3. Excellent blog thanks for sharing Run your salon business successfully by tying up with the best beauty shop in Chennai - The Pixies Beauty Shop. With tons of prestigious brands to choose from, and amazing offers we’ll have you amazed.

    ReplyDelete
  4. Awesome blog thankks for sharing 100% virgin Remy Hair Extension in USA, importing from India. Premium and original human hair without joints and bondings. Available in Wigs, Frontal, Wavy, Closure, Bundle, Curly, straight and customized color hairstyles Extensions.

    ReplyDelete
  5. Very useful blog thanks for sharing IndPac India the German technology Packaging and sealing machines in India is the leading manufacturer and exporter of Packing Machines in India.

    ReplyDelete
  6. SMBs in enterprises be it innovation or retail, are using information examination that empowers them to develop and create.Data Analytics Course in Bangalore

    ReplyDelete
  7. Great blog thanks for sharing seveninfotech is the best cctv camera wholesaler in chennai. We provide high quality cctv brands like Hikvision, Dahua, Hifocus, Essl, ZKT ECO etc. For order call - +91 95000 96878.

    ReplyDelete
  8. Cool stuff you have and you keep overhaul every one of us

    data science course

    ReplyDelete
  9. Great post i must say and thanks for the information. Education is definitely a sticky subject. However, is still among the leading topics of our time. I appreciate your post and look forward to more.

    Simple Linear Regression

    Correlation vs Covariance

    ReplyDelete
  10. I’m happy I located this blog! From time to time, students want to cognitive the keys of productive literary essays composing. Your first-class knowledge about this good post can become a proper basis for such people. nice one

    Data Science Course

    ReplyDelete
  11. I recently came across your article and have been reading along. I want to express my admiration of your writing skill and ability to make readers read from the beginning to the end. I would like to read newer posts and to share my thoughts with you.

    Data Science Training

    ReplyDelete