ABSTRACT:
The
big data era is characterized by the emergence of live content with increasing
complexities of data dimensionality and data sizes, which poses a new challenge
to emergency applications: how to timely disseminate large-scale live content
to users who are interested in. The publish/subscribe (pub/sub) model is widely
used to disseminate data because of its possibility of expanding the system to
Internet-scale size. However, existing pub/sub systems are inadequate to meet
the requirement of disseminating live content in the big data era, since their
multi-hop routing techniques and coarse-grained partitioning techniques lead to
a low matching throughput, and their upload capacities do not scale well. In
this paper, we propose a general scalable and elastic pub/sub service based on
the cloud computing environment, called GSEC. For generality, we propose a
two-layer pub/sub framework to support the dissemination with diverse data
sizes and data dimensionality. For scalability, a hybrid space partitioning
technique is proposed to achieve high matching throughput, which divides
subscriptions into multiple clusters in a hierarchical manner. Moreover,a
helper-based content distribution technique is proposed to achieve high upload
bandwidth, where servers act as both providers and coordinators to fully
explore the upload capacity of the system. For elasticity, we propose a
performance-aware provisioning technique to adjust the scale of servers to
adapt to the churn workloads. To evaluate the performance of GSEC, about 1,000
servers are deployed and hundreds of thousands of live content items are tested
in our Cloud Stack-based testbed. Extensive experiments confirm that GSEC can
linearly increase the capacities of event matching and content distribution
with the growth of servers, adaptively adjust these capacities in tens of
seconds according to the churn workloads, and significantly outperforms the
state-of-the-art approaches under various parameter settings
AIM
The aim of this paper is to propose a
two-layer pub/sub framework to support the dissemination with diverse data
sizes and data dimensionality.
SCOPE
The
Scope of this paper is GSEC can linearly increase the capacities of event
matching and content distribution with the growth of servers, adaptively adjust
these capacities in tens of seconds according to the churn workloads, and
significantly outperforms the state-of-the-art approaches under various
parameter settings.
EXISTING SYSTEM
Existing
content-based publish/subscribe systems (pub/subs) are inadequate to satisfy
the challenges of generality, scalability and elasticity. This mainly stems
from the following three reasons. Firstly, most of the pub/subs are designed
for disseminating either small-sized events or bulk content rather than give
consideration to both sides. In contrast, the data sizes of current emergency
applications may churn from tens of kilobytes to hundreds of megabytes, which
makes these pub/subs not generic. Secondly, existing space partitioning and
event routing techniques in content-based pub/subs do not scale well with the
numerous skewed subscriptions and events. Existing space partitioning
techniques lead to either unbalanced workloads among servers due to the
coarse-grained partitioning or high
memory cost due to the over fine-grained partitioning. Besides, the multi-hop
routing techniques lead to high disseminating latency. Thirdly, most pub/subs have
no financial incentives to provide elastic event matching capacities or upload
capacities according to churn workloads
DISADVANTAGES:
- The existing pub/sub systems are inadequate to meet the requirement of disseminating live content in the big data era
- Since their multi-hop routing techniques and coarse-grained partitioning techniques lead to a low matching throughput, and their upload capacities do not scale well.
PROPOSED SYSTEM
In
this paper, we propose a general scalable and elastic pub/sub service based on
the cloud computing environment, called GSEC. For generality, we propose a
two-layer pub/sub framework to support the dissemination with diverse data
sizes and data dimensionality. For scalability, a hybrid space partitioning
technique is proposed to achieve high matching throughput, which divides
subscriptions into multiple clusters in a hierarchical manner. Moreover, a
helper-based content distribution technique is proposed to achieve high upload
bandwidth, where servers act as both providers and coordinators to fully
explore the upload capacity of the system. For elasticity, we propose a
performance-aware provisioning technique to adjust the scale of servers to
adapt to the churn workloads. To evaluate the performance of GSEC, about 1,000
servers are deployed and hundreds of thousands of live content items are tested
in our Cloud Stack-based test bed. Extensive experiments confirm that GSEC can
linearly increase the capacities of event matching and content distribution
with the growth of servers, adaptively adjust these capacities in tens of seconds
according to the churn workloads, and significantly outperforms the
state-of-the-art approaches under various parameter settings.
ADVANTAGES
- GSEC can efficiently support the dissemination of diverse data sizes and data dimensionality in a general manner
- GSEC achieves more scalable performance with the increasing system sizes, and the performance-aware provisioning technique ensures continuous data dissemination service with a high performance price ratio.
SYSTEM
ARCHITECTURE:
SYSTEM
CONFIGURATION:-
Hardware Requirements
- Speed - 1.1 Ghz
- Processor - Pentium IV
- RAM - 512 MB (min)
- Hard Disk - 40 GB
- Key Board - Standard Windows Keyboard
- Mouse - Two or Three Button Mouse
- Monitor - LCD/LED
Software
requirements
- Operating System : Windows 7
- Front End : ASP.Net and C#
- Database : MSSQL
- Tool : Microsoft Visual studio
REFERENCE:
Ma,
X, Wang, Y. “A GENERAL SCALABLE AND ELASTIC CONTENT-BASED PUBLISH/SUBSCRIBE
SERVICE”, IEEE Transactions on Parallel and Distributed Systems, Volume 26, Issue 8
AUGUST 2014.
No comments:
Post a Comment