Skip to Content
author's profile photo Former Member
Former Member

Media Replication Directories on Hybris cluster nodes.

Hi All, I have one question regarding clustering, precisely "Media Replication Directories". I followed the wiki page: https://wiki.hybris.com/display/release5/Node-specific+Configurations and my nodes configuration for that look like:

local.properties node 1

cluster.id=0 cluster.0.media.read.dir=/var/www/hybris_app/media1 cluster.1.media.read.dir=/var/www/hybris_app/media2 cluster.0.media.replication.dirs=/var/www/hybris_app/media1;/var/www/hybris_app/media2

local.properties node 2

cluster.id=1 cluster.0.media.read.dir=/var/www/hybris_app/media1 cluster.1.media.read.dir=/var/www/hybris_app/media2 cluster.1.media.replication.dirs=/var/www/hybris_app/media1;/var/www/hybris_app/media2

On remote server node1 I manually created /media1, on the other server node2 I manually created /media2. The result of this configuration is than on those 2 nodes I have 2 catalogs: media1 and media2. When I use HMC on node 1 tu upload some sample file it is created in the same path in /media1 and media2. But it is not created on there's drive. The same is when using HMC on node 2: the file is created physically on media1 and media 2 (on node's2 drive). But it's not created on node's 1 drive :( What I can do is I can look for this sample files using node's 2 HMC, and I see it as they're on the same DB. But why the sample files are not created physically on each node? Maybe I don't understand the idea of media replication. But if anyone has an idea, or followed the same problem I would be more than greatfull to hear the answer.

Add a comment
10|10000 characters needed characters exceeded

Related questions

8 Answers

  • author's profile photo Former Member
    Former Member
    Posted on Aug 21, 2014 at 08:57 PM

    Hi Adriana,

    For the configuration of the media files in the cluster mode, we see three possibilities which are as follows;

    1. Using the replication.dir and configuration with enabling every cluster node to write the media to the media folder of every node For example, if you specify; media.replication.dirs=C:\media;N:\remotedir media (Backslashes need to be escaped via another backslash.) where N:\remotedir\media is a directory on a remote server, then all media created will be saved on the local hard disk (C:\media) and on the remote server (N:\remotedir\media).
    2. Using a shared driver e.g. SAN to store the medias and mount this to the data directory of every node (NFS also possible in our eyes the most performant varaity) Place from where the media filter (/medias/) is reading the media files. As path you can use something like '/usr/var/media' or 'c:\media' or '\\hostname\shared_folder' media.read.dir=${HYBRIS_DATA_DIR}/media
    3. using some sync software what replicates the medias binary files to every node's media directory
    All three solutions are used by customers successfully
    Add a comment
    10|10000 characters needed characters exceeded

  • Posted on Aug 05, 2014 at 05:18 AM

    Hi Adriana, I think you are expecting node1 to write on node2's filesystem but there is no nfs sharing configured in your case. In the examples on the wiki(windows examples), the media replication directories are shares. Basically if you want node1 to write on node2's folder you would need to create a share on node2, and mount that on node1 (if you are running a linux os) . Then your replication dir should point to that mount point. I am not sure what you are trying to accomplish by using node specific properties. Most projects needs are to share the same media folder between all clusters; in that case you'd create an nfs share, mount that to all of your nodes using the same mount point and set the media.read.dirs and media.replication.dirs to the mount point. No need to mess with node specific settings: For example, assuming /my/mediafolder was configured to point to your nfs share on each server, then you could set the following in local.properties on each node:

    media.read.dir=/my/mediafolder media.replication.dirs=/my/mediafolder

    I hope this helps.

    Add a comment
    10|10000 characters needed characters exceeded

  • author's profile photo Former Member
    Former Member
    Posted on Aug 05, 2014 at 10:33 AM

    Hi Eoin, I'm trying to find a good solution cause my Client's admins don't allow me to use NFS on their servers.

    Add a comment
    10|10000 characters needed characters exceeded

    • Former Member

      Hi Adriana,

      I don't think you can have replication or even a cluster without some form of disk sharing. You either share a single disk mounted on all nodes (a la Cristian's suggestion), or you cross-mount disks and replicate...

  • Posted on Aug 06, 2014 at 05:21 PM

    If NFS is not an option, and your goal is to share media between nodes there is another option, which I don't really prefer, but could work as a last resort. You could use rsync to push media from a back-office node to the rest of the nodes. However you'd have to be careful, rsync is one way, basically the activities that would require media uploads would have to be done on the specific node that does the rsync to the rest of the nodes. So basically depending on your use cases, you'd pin your PCM, WCMS, .... users and any automated jobs that run media imports to the node the rsync is done from. I've seen customers do this and run rsync every 5 minutes. The problem with this approach is that you'd have a 5 minute lag between the when media is published and when it's available on the other nodes. Another syncing strategy is using lsyncd which is real time sync, but I have not seen that used with hybris as of now.

    Add a comment
    10|10000 characters needed characters exceeded

  • author's profile photo Former Member
    Former Member
    Posted on Aug 21, 2014 at 09:08 PM

    In a cluster environment, media replication or a shared folder must be set to have same resource being accessed at all nodes

    Add a comment
    10|10000 characters needed characters exceeded

  • author's profile photo Former Member
    Former Member
    Posted on Aug 22, 2014 at 08:51 AM

    Hi All, yes - I set my media replication dirs as NFS shared discs spaces and cross mounted it on two nodes and it works now. Thank you.

    Add a comment
    10|10000 characters needed characters exceeded

  • author's profile photo Former Member
    Former Member
    Posted on Feb 11, 2015 at 04:45 PM

    NFS is not the only filesystem sharing option. If there are a few hybris cluster nodes, maybe you should consider using a distributed filesystem like Ceph or Gluster (probably others as well) in order to get additional benefits like fault tolerance and point-in-time snapshots while providing the single view of media files across the entire cluster required for hybris clusters.

    http://docs.ceph.com/docs/master/cephfs/

    http://www.gluster.org/community/documentation/index.php/GlusterFS_Concepts

    Add a comment
    10|10000 characters needed characters exceeded

  • author's profile photo Former Member
    Former Member
    Posted on Mar 03, 2015 at 11:50 AM

    Another option would be to use MongoDB Gridfs as a storage. We have an extension called gridfsstorage which handles it.

    Add a comment
    10|10000 characters needed characters exceeded

Before answering

You should only submit an answer when you are proposing a solution to the poster's problem. If you want the poster to clarify the question or provide more information, please leave a comment instead, requesting additional details. When answering, please include specifics, such as step-by-step instructions, context for the solution, and links to useful resources. Also, please make sure that you answer complies with our Rules of Engagement.
You must be Logged in to submit an answer.

Up to 10 attachments (including images) can be used with a maximum of 1.0 MB each and 10.5 MB total.