Rectangle 27 3

I replied on GH as well. It's sounds nice at first, but the setter is called every time, but I only need it once (that's important).

node.js - Mongoose create custom _id from other fields - Stack Overflo...

node.js mongodb mongoose
Rectangle 27 2

Is this what you are looking for?

You could define a function to create the _id before the model is saved, as in:

If this middleware is called after Mongoose creates the _id by default (my guess is it's not), you could tell Mongoose to not create an _id, with the _id option.

Well, one problem is the missing documentation for the init event (click it on the first of your links)

Yes, but where's the docu for pre('init')? Does it fire when the contructor is called or when the Schema is ready or whatever?

@Prinzhorn the init method is private which is whats causing that link to not work right. Click the little "private" checkbox at the bottom of the page to see all the private methods. The init method is called internall after a document is returned from the db.

node.js - Mongoose create custom _id from other fields - Stack Overflo...

node.js mongodb mongoose
Rectangle 27 15

You initially had a field called name in your schema, that was set to unique.

How do I know? Because of the error telling me so:

duplicate key error index: **iotdb.users.$name_1**

You renamed the field to username, but didn't remove the old index. By default, MongoDB will set the value of a non-existent field to null in that case.

If a document does not have a value for the indexed field in a unique index, the index will store a null value for this document. Because of the unique constraint, MongoDB will only permit one document that lacks the indexed field.

To solve this, you need to remove the index for the renamed name field.

you are right. i deleted that manually db.users.dropIndex({"name":1}) and ran the post again then it is working like a charm!

node.js - Mongoose - caused by :: 11000 E11000 duplicate key error ind...

node.js mongodb express mongoose
Rectangle 27 6

Typically the default value ObjectId is more ideal for the _id. Here, in this situation you can either override the default _id or you can have your own field for id(like userId in your case).

Use a separate counters collection to track the last number sequence used. The _id field contains the sequence name and the seq field contains the last value of the sequence.

Insert into the counters collection, the initial value for the userid:

db.counters.insert(    {
      _id: "userid",
      seq: 0    } )

Create a getNextSequence function that accepts a name of the sequence. The function uses the findAndModify() method to atomically increment the seq value and return this new value:

function getNextSequence(name) {
   var ret = db.counters.findAndModify(
            query: { _id: name },
            update: { $inc: { seq: 1 } },
            new: true

   return ret.seq;
     _id: getNextSequence("userid"),
     name: "Sarah C."

     _id: getNextSequence("userid"),
     name: "Bob D."

This way you can maintain as many sequences as you want in the same counter collection. For the upsert issue, check out the Optimistic Loop block in this link Create an auto-increment sequence field.

The second approach is to use a mongoose middleware like mongodb-autoincrement.

Can you please tell me this getNextSequence() function you made it in mongodb(procedure) or on your server ??

@Sudhansu you can create it at the server level as a database utility function.

node.js - MongoDB: handling auto-incrementing model id's instead of Mo...

node.js mongodb mongoose
Rectangle 27 145

You can change the default behavior at the schema definition level using the select attribute of the field:

password: { type: String, select: false }

Then you can pull it in as needed in find and populate calls via field selection as '+password'. For example:

Users.findOne({_id: id}).select('+password').exec(...);

Great. Can you provide an example on who to add it in find? Assuming I have: Users.find({id: _id}) where should I add the "+password+?

Is there a way to apply this to the object passed to save() callback? Such that when I save a user profile the password isn't included in the callback parameter.

This is by far the best answer in my opinion. Add this once, and it is exclude. Much better than adding a select or exclude option to every query.

This should be the accepted answer!

node.js - How to protect the password field in Mongoose/MongoDB so it ...

node.js mongodb express mongoose
Rectangle 27 7

Yes, it can be done. Multiple documents with the field set to null or not defined, while enforcing unique "actual" values.

  • When you field has an actual value, it should be of a specific type (e.g, always a string).

To supplement @Nolan's answer, starting with MongoDB v3.2 you can use a partial unique index with a filter expression.

The partial filter expression has limitations. It can only include the following:

  • equality expressions (i.e. field: value or using the $eq operator),
$exists: true

This means that the trivial expression {"yourField"{$ne: null}} cannot be used.

always uses the same type

{ field: { $type: <BSON type number> | <String alias> } }

You can specify it in a mongoose schema:

const UsersSchema = new Schema({
  name: {type: String, trim: true, index: true, required: true},
  email: {
    type: String, trim: true, index: {
      unique: true,
      partialFilterExpression: {email: {$type: 'string'}}

or directly add it to the collection (which uses the native node.js driver):

User.collection.createIndex("email", {
  unique: true,
  partialFilterExpression: {
    "email": {
      $type: "string"
    "email": 1
  }, {
    unique: true,
    partialFilterExpression: {
      "email": {
        $type: "string"
  function (err, results) {
    // ...
  "email": 1
}, {
  unique: true, 
  partialFilterExpression: {
    "email": {$type: "string"}

This will allow inserting multiple records with a null email, or without an email field at all, but not with the same email string.

Awesome answer. You're a savior.

Thanks partialFilterExpression did the trick

node.js - mongoDB/mongoose: unique if not null - Stack Overflow

mongodb node.js mongoose
Rectangle 27 10

Largely speaking here, $project relies on the "absolute path" to the field property in the document on the "right hand" side. Shortcuts such as 1 are just for where that element is actually the top level of the document.

Also you need to be able to retain fields when you $group, so this is where you use various grouping operators such as $first and $addToSet or $push to keep the information you are puling from the inner array. And you must $unwind twice here as well since you are combining "types" across documents, and you do not want just the $first in this case.

    { "$unwind": "$products" },
    { "$unwind": "$products.types" },
    { "$group": {
        "_id": "$",
        "active": { "$first": "$" },
        "types": { "$addToSet": "$products.types" },
        "quantity": { "$sum": 1 }
    { "$project": {
        "_id": 0,
        "name": "$_id",
        "active": 1,
        "types": 1,
        "quantity": 1
],function(err,results) {


node.js - Mongodb Trying to get selected fields to return from aggrega...

node.js mongodb mongoose aggregation-framework
Rectangle 27 3

You have a unique constraint on registerinfos.username and you're trying to save a document that has a value for that field that already exists in the database. I know this because that's what it says in the exception ;) It has nothing to do with _id values.

His original post was vague, that's why I originally thought about the _id values since I thought he was updating and it was throwing the exception but now, with the code, he is actually creating a new row.

Fair enough. The updated post clarified things I'm sure ;)

node.js - I am getting a duplicate key error index: when I am trying t...

node.js mongodb express mongoose
Rectangle 27 187

Mongoose uses the mongodb-native driver, which uses the custom ObjectID type. You can compare ObjectIDs with the .equals() method. With your example, results.userId.equals(AnotherMongoDocument._id). The ObjectID type also has a toString() method, if you wish to store a stringified version of the ObjectID in JSON format, or a cookie.

ObjectID = require("mongodb").ObjectID
results.userId instanceof ObjectID

If you're already using mongoose you can just require('mongoose').mongo.ObjectID so you don't have to list any additional dependencies

node.js - Comparing mongoose _id and strings - Stack Overflow

node.js mongodb mongoose
Rectangle 27 1

Try the following aggregation pipeline, it extracts the $year, $month, $dayOfMonth from your date and then use those as the group by key. Also, the logic for checking if the field exists and assigning it a value should ideally be done in the $project pipeline as this enables you to easily debug your pipeline operation and code is readable:

var match = {};
match["products.totalprice"] = {$exists:true};

var project = {};
    project["_id"] = 0;        
    project["line"] = {
        "year": { "$year": "$products.closedate" },
        "month": { "$month": "$products.closedate" },
        "day": { "$dayOfMonth": "$products.closedate" }
    project["total"] = {
        "$ifNull": [ "$products.totalprice", 0 ]

    {$unwind: "$products"},
    {$match: match},
    {$project: project},
    {$group: {
        _id: "$line",
        total: { $sum: "$total" }

], function(err, docs){

The is the $unwind property, otherwise it is trying to sum together arrays returned by $total.

The project["line"] statement doesn't appear to work (doc returns undefined) but without that, the totalpriceExists seems to be working. Doing some testing...

This is returning the total number of records matched by date, but not the sum of the actual totalprice field.

I updated your answer. This works, except for the group by year/month/day fields. That's not part of the OP, BUT I would actually like to get that part working, it's very useful.

node.js - Mongoose aggregate $sum returning zero - Stack Overflow

node.js mongodb mongoose aggregation-framework
Rectangle 27 50

Given you're using Mongoose, you can use 'virtuals', which are essentially fake fields that Mongoose creates. They're not stored in the DB, they just get populated at run time:

// Duplicate the ID field.
    return this._id.toHexString();

// Ensure virtual fields are serialised.
Schema.set('toJSON', {
    virtuals: true

Any time toJSON is called on the Model you create from this Schema, it will include an 'id' field that matches the _id field Mongo generates. Likewise you can set the behaviour for toObject in the same way.

You can abstract this into a BaseSchema all your models then extend/invoke to keep the logic in one place. I wrote the above while creating an Ember/Node/Mongoose app, since Ember really prefers to have an 'id' field to work with.

Schema.set('toObject', { virtuals: true })

This keeps the _id field. I prefer the transform solution instead, the one provided by shakinfree worked for me.


node.js - MongoDB: output 'id' instead of '_id' - Stack Overflow

mongodb node.js mongoose
Rectangle 27 1

Firstly, the idea of creating the index on the fields seem alright. However, creating index on all possible fields used in the query doesn't look correct.

All the queries listed in the post contain the four fields. If you create a compound index on all these 4 fields, I think it should be sufficient.

Also, you are defining the index at the schema level. When the application starts up, Mongoose automatically calls ensureIndex for each defined index in your schema. This could cause some performance impact.

You need to consider the following:-

In this options, the indexes are independent. MongoDB will always end up using an index (i.e. one of the indexes).

2) Compound index i.e. creating one index with four fields

db.collection.createIndex({field1 : 1, field2 : 1, field3 : 1, field4 :1});

Go for this option if you would have field1 always.

If you don't have field1 in any scenario, the MongoDB wouldn't use index.

I am going to make collection at once by running a script. So performance during inserting data is not an issue. Yes, I will have four fields in all the cases and the other fields comes in different cases. Does it have any effect on the performance If I don't add index over the age and gender field ? and I will have 250,0000 documents on which I have to query.

After filtering the data by four fields, I presume you wouldn't have 250,0000 documents. You would have small subset of total, normally MongoDB should be ok to handle that without index. I have never seen all the fields in the document added in the index. Definitely, this is not correct.

node.js - Indexing in mongoose documents - Stack Overflow

node.js mongodb mongoose mongoose-schema
Rectangle 27 2

It seems like you're treating Mongo like an relational database (separating all these fields and bringing them together with a query). What you should do is rebuild your Schema. For example:

var CategorySchema = new Schema({
   category_name: String,

var subCategorySchema = new Schema({
   category_name: String

This way when you need to query the collection it's a simple

db.find({category_name: "name of the category"}, function(){})

to get everything you need.

Just in case: you can add the sub categories to the array with simple updates. Read this for more info.

Had to modify this a little bit to suit my needs but overall it was what I needed. Thanks!

javascript - How to group records by a field in Mongoose? - Stack Over...

javascript node.js mongodb mongoose nosql
Rectangle 27 2

It seems to me that the document was too big, actually I was able to save it after removing one of its fields. The weird thing is that mongoose should catch that with "Error: Document exceeds maximum allowed bson size of XXX bytes" exception, but that was never thrown.

node.js - MongoError: write EPIPE - Stack Overflow

node.js mongodb mongoose epipe
Rectangle 27 1

There is no native function or way to list all fields of a collection because MongoDB is "schemaless". You need to iterate all the documents from the collection and get all the field names. Or simply get the field names from the results of the query you are displaying (like Robomongo).

You can try a MapReduce like the suggest in this post.

node.js - How to get all fields name with nodeJS - Stack Overflow

node.js mongodb mongoose
Rectangle 27 1

There is no native function or way to list all fields of a collection because MongoDB is "schemaless". You need to iterate all the documents from the collection and get all the field names. Or simply get the field names from the results of the query you are displaying (like Robomongo).

You can try a MapReduce like the suggest in this post.

node.js - How to get all fields name with nodeJS - Stack Overflow

node.js mongodb mongoose
Rectangle 27 1

Use a sparse unique index

If a document does not have a value for a field, the index entry for that item will be null in any index that includes it. Thus, in many situations you will want to combine the unique constraint with the sparse option. Sparse indexes skip over any document that is missing the indexed field, rather than storing null for the index entry.

db.collection.createIndex( { a: 1, b: 1 }, { unique: true, sparse: true } )

Thanks man! this is what i was looking for

javascript - Mongoose field not required but unique - Stack Overflow

javascript node.js mongoose
Rectangle 27 154

It's simple, you can define this in the subschema :

var mongoose = require("mongoose");

var subSchema = mongoose.Schema({
    //your subschema content
},{ _id : false });

var schema = mongoose.Schema({
    // schema content
    subSchemaCollection : [subSchema]

var model = mongoose.model('tablename', schema);

will this skip _id fields even in the subSchema collection, or only in the case where the subSchema is being used to embed as array of sub-document items? I ask this particularly because of my own question on SO today.

I use two levels of nested sub-schema collections. In other words, I have a subSchema collection similar to your example. Within that, I use another different sub-schema collection. I want only the first level sub-schema model instances to not use ids, but the second level (nested) sub-schema model instances need to have ids. When I use your solution, that is, specifying { _id: false }, both levels of sub-schema are without ids. Any way to work around this behavior?

{ _id : true }
let studentSchema = new Schema({ studentId: { type: ObjectId, ref: Student.modelName }, performance: [performanceSchema] }, { _id: false });
let studentSchema = new Schema({ _id: false, id: false, studentId: { type: ObjectId, ref: Student.modelName }, performance: [performanceSchema] });
_id: false
id: false

node.js - Stop Mongoose from creating _id property for sub-document ar...

node.js mongodb mongoose subdocument
Rectangle 27 76

You can create sub-documents without schema and avoid _id. Just add _id:false to your subdocument declaration.

var schema = new mongoose.Schema({
      field :{type:String}

This will prevent the creation of an _id field in your subdoc. Tested in Mongoose 3.8.1

node.js - Stop Mongoose from creating _id property for sub-document ar...

node.js mongodb mongoose subdocument
Rectangle 27 3

If manually created _id is not Object By default, mongoose assumes that _id is an object ID. If you want to be able to get the manually create IDs, you need to specify that in your schema.

If _id is of Number then you have to explicitly specify in schema.

schema= new Schema({
            _id: Number

var Thread = mongoose.model('threads', schema);

node.js - Include Mongodb "_id" field in projection with Mongoose - St...

node.js mongodb express mongoose