Successfully reported this slideshow.
We use your Verifieduser profile and activity data to personalize ads and to show you more relevant ads.
Pinot
Kishore Gopalakrishna
Tuesday, August 18, 15
Agenda
• Pinot @ Verifieduser - Current
• Pinot - Architecture
• Pinot Operations
• Pinot @ Verifieduser - Future
Tuesday, August ...
WVMP
Tuesday, August 18, 15
Slice and Dice Metrics
Tuesday, August 18, 15
Pinot @ Verifieduser
Customers Members Internal tools
Tuesday, August 18, 15
• 100B documents
• 1B documents ingested per day
• 100M queries per day
• 10’s of ms latency
• 30 tables in prod, 250 * 3 ...
Key features
SQL-like
interface
Columnar
storage and
indexing
Real-time
data load
Tuesday, August 18, 15
(S)QL: Filters and Aggs
SELECT count(*)
FROM companyFollowHistoricalEvents
WHERE entityId = 121011 AND
'day' >= 15949 AND ...
(S)QL: Group By
SELECT count(*)
FROM companyFollowHistoricalEvents
WHERE entityId = 121011 AND
'day' >= 15949 AND 'day' <=...
(S)QL: ORDER BY and LIMIT
SELECT *
FROM companyFollowHistoricalEvents
WHERE entityId = 121011 AND
entityId = 1000 AND
acti...
Whats not supported
• JOIN: unpredictable performance
• NOT A SOURCE OF TRUTH
• Mutation
Tuesday, August 18, 15
Pinot
• Data flow
• Query Execution
• How to use/operate
• Pinot @ Verifieduser - Future
Tuesday, August 18, 15
Broker Helix
Real
time
Historical
Kafka Hadoop
Pinot
Architecture
Queries
Raw
Data
Tuesday, August 18, 15
Pinot
• Pinot segments
Tuesday, August 18, 15
Pinot Segment layout: Columnar storage
Tuesday, August 18, 15
Pinot Segment layout: Sorted Forward Index
Tuesday, August 18, 15
Pinot Segment layout: Other techniques
• Indexes: Inverted index, Bitmap, RoaringBitmap
• Compression: Dictionary Encoding...
Data aware
pre-computation
Star tree Index
Tuesday, August 18, 15
Pinot
• Query Execution
Tuesday, August 18, 15
Pinot Query Execution: Distributed
Servers
S1
S3
S2
S1
S3
S2
Helix
Brokers
Tuesday, August 18, 15
Pinot Query Execution: Distributed
Servers
1.Query
S1
S3
S2
S1
S3
S2
Helix
Brokers
Tuesday, August 18, 15
Pinot Query Execution: Distributed
Servers
1.Query
S1
S3
S2
S1
S3
S2
Helix
2. Fetch routing table from HelixBrokers
Tuesda...
Pinot Query Execution: Distributed
Servers
1.Query
S1
S3
S2
S1
S3
S2
Helix
2. Fetch routing table from HelixBrokers
3. Sca...
Pinot Query Execution: Distributed
Servers
1.Query
S1
S3
S2
S1
S3
S2
Helix
2. Fetch routing table from HelixBrokers
3. Sca...
Pinot Query Execution: Distributed
Servers
1.Query
S1
S3
S2
S1
S3
S2
Helix
2. Fetch routing table from HelixBrokers
3. Sca...
Pinot Query Execution: Distributed
Servers
1.Query
S1
S3
S2
S1
S3
S2
Helix
2. Fetch routing table from HelixBrokers
3. Sca...
Pinot Query Execution: Single Node Architecture
EXECUTION ENGINE
INVERTED
INDEX
BITMAP
INDEX
COLUMN FORMAT
PLANNER
Tuesday...
Pinot Query Execution: Single Node Architecture
SELECT
campaignId,
sum(clicks)
FROM Table A
WHERE
accountId = 121011
AND
'...
Pinot
• Operations
Tuesday, August 18, 15
Cluster Management: Deployment
Helix
Brokers
Servers
• Brokers and Servers register themselves in Helix
• All servers star...
On boarding new use case
Helix
Brokers
Servers
XLNT XLNT
XLNT
Create Table
command
Controller
XLNT
XLNTTag
Servers
TableNa...
Segment Assignment
Servers
S3
S2
S1
Upload Segment S2
S1
S3
S2
S1
S3
Helix
Brokers
Copies
TableName
2
XLNT_T1
Controller
T...
• AUTO recovery mode: Automatically redistribute
segments on failure/addition of new nodes
• Custom mode: Run in degraded ...
Pinot vs Druid
Druid Pinot
Architecture
Realtime + Offline,
Realtime only
Realtime + Offline
Realtime only -> consistency is...
• Documentation & tooling
• In progress - consistency among real time replicas.
• Improve cost to serve - leverage SSD, pa...
Thank You
30
Tuesday, August 18, 15
Upcoming info
Loading in …5
×

Slice and Dice Metrics Tuesday, Pinot: Realtime Distributed OLAP datastore

708,814 views

Published on

Slice and Dice Metrics
Tuesday, August 18, 15

Published in: Technology
  • EFFECTIVE AND POWERFUL LOVE SPELL,LOTTERY SPELL,MONEY SPELL CASTER DR GBOJIE  +2349066410185   With  joy in my heart I want to use this medium to tell the world about Lord gbojie who helped me in getting my lover back with his powerful spell, my ex and i was having misunderstanding which led to our breakup and i went to beg him several times to please forgive and accept me back because i know i offended him but each time i went i always feel more deeply in pain and agony because he always walk out on me and would not want to listen to what i have to tell but on a faithful day as i was browsing i came across  a testimony of a woman whose problem was more than mine and yet Lord gbojie helped her with his powerful and effective spell so i was happy and also ed Lord gbojie for help via email and then i told him my story but the only thing he said was that i will wipe you tear with my spell so lucky for me everything want well just as he promised and right now i have got my lover back and we are both living happily. there is nothing Lord gbojie can not do with his spell and just as promise my self i will keep testifying on the Internet of how Lord gbojie helped me. Are your problem greater that mine or less i give you 100% guarantee that Lord gbojie will put an end to it with his powerful spell, Lord gbojie for help you can reach him no his email : [email protected] or [email protected] you can also call him or whatsapp  +2349066410185  website : http://gbojiespiritualtemple.website2.me/  
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Ngoài ra, giá căn hộ Opal Garden chỉ từ 1,4 tỷ đồng/căn 2 phòng phòng chỉ từ khi mới ra mắt năm 2016. Nhưng theo khảo sát của bất động sản top sau gần 2 năm, giá căn hộ đã tăng lên mức 2,2 tỷ đồng (tăng 60%) cho căn 58m2 và 3,3 tỷ đồng loại 84 – 96 m2 thiết kế 3 phòng ngủ. #batdongsantop #bất_động_sản_top https://batdongsantop.com/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • My life Was devastated! But All Thanks To Dr happy Who Brought Back My girlfriend after cheating on me and ask for a breakup, Alexis scorpio from FL Usa. I don't believe at first i try to get back with her but all she told me was she’s with someone else. that she is no longer interested in marrying me at that point i was heart broken coz i love my girlfriend so much that i could not let go off her all of a sudden she left me, i really love her and never can imagine my life without her. not until i came across a powerful real spell caster Dr happy who promise me 24hours urgent love spell to get back with my girlfriend, good twenty-four {24} hours. hmm-mm, it was a good night time at 11:pm within the days that Dr happy told me that my girlfriend will be back, at first i heard the bell rings getting close to my door i heard someone saying honey!!!, it sound familiar i opened the door and i saw my girlfriend standing and weeping in front of me. i was not surprised because its all i have been praying for her to come back home. Guess what 2 weeks after she noticed her system and her body temperature was changed and i took her to clinic for check up and the doctor told me that there is life in her which means she was pregnant i really wants to use this opportunity to thanks Dr happy so much and the love page that directed me to Dr happy if you have any problem getting your ex back, or predicament that is worse or exactly like this you have been into, Dr happy on [email protected] Whatsapp/cal +2348133873774 visit his website...happylovespell2.webnode.com/ about his love spell Blogs site.http://happylovespel.blogspot.com.ng/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Unlock Her Legs - How to Turn a Girl On In 10 Minutes or Less... ♣♣♣ http://scamcb.com/unlockher/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Start working at home with Google! Just work for few hours and have more time with friends and family. I earn up to $4500 per week. It's a great work at home opportunity. I can't believe how easy it was once I tried it out. Visit for more details.☛☛☛☛☛☛☛☛☛☛☛☛☛☛☛☛☛☛☛☛☛☛☛☛ www.Buzz90.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

×