Best User File Management using math.fmod

Introduction

Where is the best place to store images on your Web site? If you have a huge content application and store all your images in a single folder called “images”.you might land up you application in “BLack Hole” situation..

Storing all the images of the application in just one folder have many problems like.

1. The “black hole”.
2. Name conflict.
3. Access control.
4. Scalability.
5. Performance.

The Black Hole:

“Black hole” is an appropriate name for the  folder where all the files are stored , because files go into this folder but never come out. Over time, more and more new files are added to the “Black Hole” folder. However, when users are deleted files that are no longer useful are left in the folder. Why? The primary reason is because files stored in the  folder may be  shared resource and it’s hard to tell if a given file is being used or not . So the “Black Hole” folder is left unmanaged and becomes a stale collection of any file that has ever been used on the application.

Name Conflict:

One of the first problems content management encounters is file name conflict, when a user uploads a files with a file name that already exists in the repository.Most of the management systems append ‘_1′ , ‘_2′ to the file name to remove the conflicts but that looks very ugly for the system as well as users.The more file you have in a single folder, the greater the chances that a name conflict will occur.

Access control:

Your management team may control access to the  documents/files .Using Singe folder to store all the files will cause the access control conflicts. Content authors are permitted to delete, rename or replace images in that folder. Content creators can manipulate images used by documents that they don’t have access to, and can even change the branding of the application.

Scalability:

Most application provide an file library interface that either displays a list box or thumbnails of files in the library. If the  folder contains hundreds or thousands of files , it becomes impractical for applications to present these images in this way to content authors and for content authors to find the appropriate files.

Performance:

File systems don’t perform well when a large number of files are stored in a single folder. The file management library may open slowly for users, and if the  library displays metadata about file such as dimensions, file size or date last modified, this slows performance even further. Large  libraries also require more data to be transmitted over the network, which can be resource intensive and slow. 

A better way to manage Files

In this article, we will limit the discussion to storing images on the file system, although this approach can be adapted to storing images in the database as well.

The solution to the problems created by using a single folder can be addressed by creating a different folder on the file system for each User/Object. The folder name should be the ID of the User/Object in the application and all the files used by a given User/Object should be stored in this folder. Although these folders can be created anywhere on your site, the examples in this article will assume all these folders are sub-folders stored in a folder called “User_Files” of the root of the application.

As discussed previously, file systems don’t perform well when a large number of files are stored in a single folder. The same is true for sub-folders. The solution is therefore to create grouping folders that will significantly reduce the number of immediate sub-folders in any given folder.

If your system uses sequentially numbered User/Object IDs, then you can use a math.fmod(x, y) arithmetic operation to create the name of the grouping folders. (x is your User/Objects ID and Y is your max folder limit number lets say 999). fmod operations return the remainder from a division of two numbers. So to get 999 grouping folders, simply fmod the User/Object ID by 999, which will produce numbers from 0 to 999, which you can use to name your grouping folders. The following illustrates how these folders will be nested:

• User_Files
  • /1     #fmod(objectID,999)
    • /1000    #objectID
      • /my_resume.pdf   #FileName
      • /project_details.doc
    • /1256
      • /NDA.pdf
  • /337
    • /2335
      • /company_policy.docx
      • /designs.psd
    • /1257
      • /my_image.gif
    • /1513
  • /338
    • /2336
      • /logos.jpg
      • /product1.tiff
      • /HomePage.jpeg

Conclusion

Storing files  in separate folders solves many of the problems caused by using a single folder. This approach is simple and reliable and can  be used for attachments (documents like PDF, Microsoft Word, Zip files, etc.). In fact, you can store both images and attachments in the same folder.

Image may be NSFW.
Clik here to view.

Image may be NSFW.
Clik here to view.