A SCALABLE AND RELIABLE MATCHING SERVICE FOR CONTENT-BASED PUBLISH/SUBSCRIBE SYSTEMS

ABSTRACT

Characterized by the increasing arrival rate of live content, the emergency applications pose a great challenge: how to disseminate large-scale live content to interested users in a scalable and reliable manner. The publish/subscribe (pub/sub) model is widely used for data dissemination because of its capacity of seamlessly expanding the system to massive size. However, most event matching services of existing pub/sub systems either lead to low matching throughput when matching a large number of skewed subscriptions, or interrupt dissemination when a large number of servers fail. The cloud computing provides great opportunities for the requirements of complex computing and reliable communication. In this paper, we propose SREM, a scalable and reliable event matching service for content-based pub/sub systems in cloud computing environment. To achieve low routing latency and reliable links among servers, we propose a distributed overlay Skip Cloud to organize servers of SREM. Through a hybrid space partitioning technique HPartition, large-scale skewed subscriptions are mapped into multiple subspaces, which ensures high matching throughput and provides multiple candidate servers for each event. Moreover, a series of dynamics maintenance mechanisms are extensively studied. To evaluate the performance of SREM, 64 servers are deployed and millions of live content items are tested in a CloudStack testbed. Under various parameter settings, the experimental results demonstrate that the traffic overhead of routing events in Skip Cloud is at least 60% smaller than in Chord overlay, the matching rate in SREM is at least 3.7 times and at most 40.4 times larger than the single-dimensional partitioning technique of Blue Dove. Besides, SREM enables the event loss rate to drop back to 0 in tens of seconds even if a large number of servers fail simultaneously.

AIM

The aim of this paper is To achieve low routing latency and reliable links among servers, we propose a distributed overlay Skip Cloud to organize servers of SREM..

SCOPE

The scope of this paper tend to evaluate the performance of SREM, 64 servers are deployed and millions of live content items are tested in a CloudStack testbed.

EXISTING SYSTEM

Recently, cloud computing provides great opportunities for the applications of complex computing and high speed communication , where the servers are connected by high speed networks, and have powerful computing and storage capacities. A number of pub/sub services based on the cloud computing environment have been proposed, such as Move,BlueDove and SEMAS . However, most of them can not completely meet the requirements of both scalability and reliability when matching large scale live content under highly dynamic environments. This mainly stems from the following facts:1) Most of them are inappropriate to the matching of live content with high data dimensionality due to the limitation of their subscription space partitioning techniques, which bring either low matching throughput or high memory overhead. 2) These systems adopt the one-hop lookup technique among servers to reduce routing latency. In spite of its high efficiency, it requires each dispatching server to have the same view of matching servers. Otherwise, the subscriptions or events may be assigned to the wrong matching servers, which brings the availability problem in the face of current joining or crash of matching servers. A number of schemes can be used to keep the consistent view, like periodically sending heartbeat messages to dispatching servers or exchanging messages among matching servers. However, these extra schemes may bring a large traffic overhead or the interruption of event matching service.

DISADVANTAGES

1. To disseminate large-scale live content to interested users in a scalable and reliable manner
2. Most event matching services of existing pub/sub systems either lead to low matching throughput when matching a large number of skewed subscriptions, or interrupt dissemination when a large number of servers fail

PROPOSED SYSTEM

In this paper propose a scalable and reliable matching service for content-based pub/sub service in cloud computing environments, called SREM. Specifically, we mainly focus on two problems: one is how to organize servers in the cloud computing environment to achieve scalable and reliable routing. The other is how to manage subscriptions and events to achieve parallel matching among these servers. a distributed overlay protocol, called Skip Cloud, to organize servers in the cloud computing environment. Skip Cloud enables subscriptions and events to be forwarded among brokers in a scalable and reliable manner. Also it is easy to implement and maintain. To achieve scalable and reliable event matching among multiple servers, we propose a hybrid multi-dimensional space partitioning technique, called HPartition. It allows similar subscriptions to be divided into the same server and provides multiple candidate matching servers for each event. Moreover, it adaptively alleviates hot spots and keeps workload balance among all servers.

ADVANTAGES

1. Through a hybrid multi-dimensional space partitioning technique, SREM reaches scalable and balanced clustering of high dimensional skewed subscriptions, and each event is allowed to be matched on any of its candidate servers
2. Extensive experiments with real deployment based on a Cloud Stack testbed are conducted, producing results which demonstrate that SREM is effective and practical, and also presents good workload balance, scalability and reliability under various parameter settings.

 SYSTEM ARCHITECTURE

SYSTEM CONFIGURATION

HARDWARE REQUIREMENTS:-

·                Processor          -   Pentium –III

·                Speed                -    1.1 Ghz

·                RAM                 -    256 MB(min)

·                Hard Disk         -   20 GB

·                Floppy Drive    -    1.44 MB

·                Key Board         -    Standard Windows Keyboard

·                Mouse               -    Two or Three Button Mouse

·                Monitor             -    SVGA

SOFTWARE REQUIREMENTS:-

·                Operating System       : Windows  7                                    

·                Front End                  : JSP AND SERVLET

·                Database                  : MYSQL

·                Tool                           :NETBEANS

REFERENCE

Yijie Wang ; Xiaoqiang Pei, Xingkong Ma “A SCALABLE AND RELIABLE MATCHING SERVICE FOR CONTENT-BASED PUBLISH/SUBSCRIBE SYSTEMS” IEEE Transactions on Cloud Computing, Volume 3 ,  Issue 1 July 2014.